Arize AI
Search…
Multi-label Classification

Multi-label Classification Models

Multi-label models are a variant of a classification models where an inference can be assigned multiple labels, often with a probability score. Arize supports logging the prediction category and the prediction score for each of the labels by passing the label category as a tag in addition to the prediction label variable.
Field
Data Type
Arize Field
Prediction Label
List of Strings
prediction_label (one per inference exploded on the labels)
Prediction Score (if scored)
List of Floats
prediction_score (one per inference exploded on the labels)
Actual Label
List of Strings
actual_label (one per inference exploded on the labels)
Actual Score (if prediction score was logged)
List of Floats
actual_score (one per inference exploded on the labels)
Tags
String
tag e.g (tag.predicted_label = label)
(one per inference exploded on the labels)

Example ML Use Case

Product Services
  • Prediction Labels: Which service packages will this consumer buy?
    • Labels = [Sports, Movies, Learning, Games]
  • Prediction Score: How likely is this product to be purchased?
    • Scores = [0.9, 0.8, 0.2, 0.5]
  • Actual Label: "Which service packages DID this consumer buy?"
    • Labels = [Sports, Movies]

Performance Metrics Supported

If you include score values, this model type will support both classification and regression metrics.
Field Ingested
Available Metrics
Prediction & Actual Label
Prediction & Actual Score
  • MAPE: Mean Absolute Percent Error
  • MAE: Mean Absolute Error
  • RMSE: Root Mean Square Error
  • MSE: Mean Squared Error
  • R^2: R-Squared or Coefficient of Determination
  • Mean Error: Average Error between Predictions & Actuals
Both Labels and Scores for Predictions and Actuals
All of the above metrics plus:

Code Example: Logging to Arize

To log multi-label models into Arize, you'll need to explode a prediction into N predictions where N is the number of Labels being predicted and add the name of the label as a tag.
# One inference yield 4 independent probabilities: Click, Save, Hide, Buy
record =
{
"prediction_id":"sample_pred_id",
"predictions":{
"click":0.67,
"save":0.32,
"hide":0.13,
"buy":0.07
},
"actuals":[
"click",
"hide"
]
}
# Log 4 separate events, 1 for each predicted label
for pred_label in record['predictions']:
# For clarity, extracting values into variables
predicion_id = f"{record['prediction_id']}_{pred_label}"
prediction_label = pred_label
prediction_score = record['predictions'][pred_label]
actual_label = pred_label if pred_label in record['actuals'] else None
actual_score = 1.0 if pred_label in record['actuals'] else None
tag = pred_label
response = arize.log(
model_id='sample-model-1',
prediction_id=predicion_id,
model_type=ModelTypes.SCORE_CATEGORICAL,
prediction_label=prediction_label,
prediction_score=prediction_score,
actual_label=actual_label,
actual_score=actual_score,
tags = {'label': tag}
)
Copy link
Outline
Multi-label Classification Models
Example ML Use Case
Performance Metrics Supported
Code Example: Logging to Arize