Ask or search…
K
Links

Multi-Class Classification

How to log your model schema for multiclass classification models

Multi-Class Classification Overview

A classification model with more than two classes.

Supported Metrics

How To Log Multi-Class Data

Log multi-class classification models based on your use case
Use Case
Description
Expected Fields
Single-Label
A prediction that has 1 label i.e. A passenger can only be in EITHER economy, business, OR first-class
  • prediction scores (dictionary)
  • actual scores (dictionary, optional)
Multi-Label
A prediction that has multiple labels i.e. A song can be multiple genres such as 'pop-rock'
  • prediction scores (dictionary)
  • threshold scores (dictionary)
  • actual scores (dictionary, optional)

Single-Label Use Case

Python Pandas Batch
Python Single Record
Google Colaboratory
Single-Label Use Case Colab
Example Row
prediction_scores
actual_scores
[{"class_name": "economy_class", "score": 0.81},{"class_name": "business_class", "score": 0.42},{"class_name": "first_class", "score": 0.35}]
[{"class_name": "economy_class", "score": 1}]
Note: class economy_class has the highest prediction score and will be the prediction label
Code Example
schema = Schema(
prediction_id_column_name="prediction_id",
prediction_score_column_name="prediction_scores",
actual_score_column_name="actual_scores",
)
response = arize_client.log(
model_id='multiclass-classification-single-label-example',
model_version= "v1",
model_type=ModelTypes.MULTI_CLASS,
dataframe=example_dataframe,
schema=schema,
environment=Environments.PRODUCTION,
)
Google Colaboratory
Single-Label Use Case Single RecordColab
# Predicting likelihood of Economy, Business, or First Class
"""
example_record = {
"prediction_scores":{
"economy_class":0.81,
"business_class":0.42,
"first_class":0.35
},
"actual_scores": {
"first_class": 1
}
}
"""
prediction_label = MultiClassPredictionLabel(
prediction_scores=record["prediction_scores"],
)
actual_label = MultiClassActualLabel(
actual_scores=record["actual_scores"],
)
response = arize_client.log(
model_id="multiclass-classification-single-label-example",
model_version= "v1",
model_type=ModelTypes.MULTI_CLASS,
prediction_id=record["prediction_id"],
prediction_label=prediction_label,
actual_label=actual_label,
environment=Environments.PRODUCTION
)
Note: class economy_class has the highest prediction score and will be the prediction label

Multi-Class Prediction

Log multi-class prediction values using the MultiClassPredictionLabel object, which can be assigned to the prediction_label parameter.
Prediction scores are required to use MULTI_CLASS models in Arize.
class MultiClassPredictionLabel(
prediction_scores: Dict[str, Union[float, int]]
)

Multi-Class Actual Values

Log multi-class actual values by using the MultiClassActualLabel object, which can be assigned to the actual_label parameter.
Actual values are optional, and can be sent later via delayed actuals.
class MultiClassActualLabel(
actual_scores: Dict[str, Union[float, int]]
)
For more information on Python Single Record Logging API Reference, visit here:

Multi-Label Use Case

Python Pandas Batch
Python Single Record
Google Colaboratory
Multi-Label Use Case Colab
Example Row
prediction_scores
threshold_scores
actual_scores
[{"class_name": "jazz", "score": 0.81},{"class_name": "rock", "score": 0.42},{"class_name": "pop", "score": 0.35}]
[{"class_name": "jazz", "score": 0.5},{"class_name": "rock", "score": 0.4},{"class_name": "pop", "score": 0.6}]
[{"class_name": "rock", "score": 1}]
Note: classes jazz and rock have prediction scores > threshold scores and will be part of the prediction label.
Code Example
schema = Schema(
prediction_id_column_name="prediction_id",
prediction_score_column_name="prediction_scores",
multi_class_threshold_scores_column_name="threshold_scores"
actual_score_column_name="actual_scores",
)
response = arize_client.log(
model_id='multiclass-classification-multi-label-example',
model_version= "v1",
model_type=ModelTypes.MULTI_CLASS,
dataframe=example_dataframe,
schema=schema,
environment=Environments.PRODUCTION,
)
Google Colaboratory
Multi-Label Use Case, Single Record Colab
#predicting song genre within pop, rock, and jazz categories
"""
example_record = {
"prediction_id": "57437c5d-0cd2-48bf-aa91-379302fd0fe4",
"prediction_scores": {
"pop": 0.5044088627000001,
"rock": 0.3688503145,
"jazz": 0.1267408228
},
"threshold_scores": {
"pop": 0.50,
"rock": 0.40,
"jazz": 0.10
},
"actual_scores": {
"pop": 1,
"jazz": 1
}
"""
prediction_label = MultiClassPredictionLabel(
prediction_scores=record["prediction_scores"],
threshold_scores=record["threshold_scores"], #required for multi-label
)
actual_label = MultiClassActualLabel(
actual_scores=record["actual_class"],
)
response = arize_client.log(
model_id="multiclass-classification-multi-label-example",
model_version= "v1",
model_type=ModelTypes.MULTI_CLASS,
prediction_id=record["prediction_id"],
prediction_label=prediction_label,
actual_label=actual_label,
environment=Environments.PRODUCTION
)

Multi-Class Prediction & Threshold Values

Log multi-class prediction and threshold values using the MultiClassPredictionLabel object, which can be assigned to the prediction_label parameter.
Prediction scores are required to use MULTI_CLASS models in Arize.
class MultiClassPredictionLabel(
prediction_scores: Dict[str, Union[float, int]]
threshold_scores: Dict[str, Union[float, int]] = None #for multi-label use cases
)

Multi-Class Actual Values

Log multi-class actual values by using the MultiClassActualLabel object, which can be assigned to the actual_label parameter.
Actual values are optional, and can be sent later via delayed actuals.
class MultiClassActualLabel(
actual_scores: Dict[str, Union[float, int]]
)
For more information on Python Single Record Logging API Reference, visit here:

Inferring Labels From Uploaded Scores

To calculate metrics and visualize & troubleshoot data for multi-class models, Arize automatically infers prediction & actual labels from the scores that you upload.
Learn how each case is determined below.
Use Case
Prediction Label Determination
Actual Label Determination
Single-Label
For each prediction, the class with the highest prediction score is the prediction label
The class with an actual score of 1 is the actual label
Multi-Label
For each class, there must exist a prediction score and threshold score. If the prediction score > threshold score, the class is a part of the prediction label
Each class with an actual score of 1 is part of the actual label