Arize AI
Search…
⌃K

Binary Classification

How to log your model schema for binary classification models

Binary Classification Model Overview

Classification models have a discrete classification label. There are different binary classification variants depending on your combination of scores and labels.

Binary Classification Variants

Binary classification variants differ based on your model's score and label availability. Variants direct the performance metrics available for evaluation.
Variant
Expected Fields
Performance Metrics
prediction label, actual label
Accuracy, Recall, Precision, FPR, FNR, F1, Sensitivity, Specificity
prediction score, actual label
AUC, PR-AUC, Log Loss
prediction score, prediction label, actual label
AUC, PR-AUC, Log Loss, MAPE, MAE, RMSE, MSE, R2, Mean Error
prediction score, actual score
MAPE, MAE, RMSE, MSE, R2, Mean Error
prediction score, prediction label, actual score
AUC, Accuracy, Recall, Precision, FPR, FNR, F1, Sensitivity, Specificity
all labels and scores
Accuracy, Recall, Precision, FPR, FNR, F1, Sensitivity, Specificity, MAPE, MAE, RMSE, MSE, R2, Mean Error, AUC, PR-AUC, Log Loss

Label and Score Definitions

Depending on your model variant, you will need to input different prediction labels & scores and actual labels & scores.

Prediction Label: The classification label of this event

Is this a Fraud Transaction?
prediction_label = "not fraud"
Datatype = str | bool

Actual Label: The ground truth label

"Was this actually a fraud transaction?"
actual_label = "fraud"
Datatype = str | bool

Prediction Score: Score the likelihood of the event

How likely is this to be fraud?
prediction_score = 0.3
Datatype = float

Actual Score: The ground truth score

"Was this actually fraud?"
actual_score = 0.6
Datatype = float
For some use cases, it may be important to treat a prediction for which no corresponding actual label has been logged yet as having a default negative class actual label.
For example, consider tracking advertisement conversion rates for an ad clickthrough rate model, where the positive class is click and the negative class is no_click. For ad conversion purposes, a prediction without a corresponding actual label for an ad placement is equivalent to logging an explicit no_click actual label for the prediction. In both cases, the result is the same: a user has not converted by clicking on the ad. For AUC-ROC, PR-AUC, and Log Loss performance metrics, Arize supports treating predictions without an explicit actual label as having the negative class actual label by default. In the above example, a click prediction without an actual would be treated as a false positive, because the missing actual for the prediction would, by default, be assigned to the no_click negative class.
This feature can be enabled for monitors and dashboards via the model performance config section of your model's config page.