Drift Tracing

How To Troubleshoot Drift

Once a drift monitor triggers alerting you that your model has drifted beyond your threshold value, troubleshoot the cause of your drift in the drift tab to begin troubleshooting.

• Prediction Drift

• Feature Drift

• Change Your Baseline

• Delayed Actuals

Prediction Drift

Prediction Drift refers to a change in the prediction distribution of your model. Drift is measured using a reference data set. You'll also see details of your baseline dataset that's used as a comparison.

Begin troubleshooting by clicking on parts of your model that deviate from your threshold on the Prediction Drift curve. From there, uncover changes in your Distribution Comparison and Feature Drift in the cards below.

Distribution Comparisons for values above and below the model baseline

Feature Drift

Once you've identified problem areas in your model to investigate, narrow down on problematic areas to troubleshoot with the Feature Drift card under the Performance Drift Chart. Feature Drift refers to changes in your feature distribution. Since feature drift is very common in the real world, it can happen at any time while your model is in production, yet remain undetected.

The Feature Drift card highlights the most impactful features degrading your model and uncovers granular drift information per feature.

Click into each feature name to look at the feature drift over time chart and the feature's distribution comparison to easily identify missing values, problematic slices, and areas to improve.

Pro Tip: Learn how to configure the visualization of a feature's distribution comparison here.

Change Your Baseline

Your baseline can help indicate why your model has drifted.

To detect drift over time, set your baseline using training data to identify how your model changes between your features, predictions, and actuals. To detect short-term drift, set your baseline using historical production data (i.e. 2 weeks).

Production data against training data baseline:

This drift indicates that the data used to build the model has shifted from the data now seen in production.

Current production data against recent production data baseline:

This drift indicates high variance in production data distributions, which could indicate model performance degradation, data quality issues, or noisy features.

Delayed Actuals

It is common to receive delayed actuals (aka ground truth) for your production model. In this situation, use proxy metrics to evaluate model performance. The most common proxy metrics for model performance are:

• Feature Drift

• Prediction score/class distribution drift

• Change in prediction score/class

Many model issues can be resolved by retraining your model with targeted training data uncovered in Arize troubleshooting workflows (such as identifying the culprit of drift)!