Drift Tracing
Last updated
Was this helpful?
Last updated
Was this helpful?
Once a drift monitor triggers alerting you that your model has drifted beyond your threshold value, troubleshoot the cause of your drift in the drift tab to begin troubleshooting.
Prediction Drift refers to a change in the prediction distribution of your model. Drift is measured using a . You'll also see details of your baseline dataset that's used as a comparison.
Begin troubleshooting by clicking on parts of your model that deviate from your threshold on the Prediction Drift curve. From there, uncover changes in your Distribution Comparison and Feature Drift in the cards below.
Once you've identified problem areas in your model to investigate, narrow down on problematic areas to troubleshoot with the Feature Drift card under the Performance Drift Chart. Feature Drift refers to changes in your feature distribution. Since feature drift is very common in the real world, it can happen at any time while your model is in production, yet remain undetected.
The Feature Drift card highlights the most impactful features degrading your model and uncovers granular drift information per feature.
Click into each feature name to look at the feature drift over time chart and the feature's distribution comparison to easily identify missing values, problematic slices, and areas to improve.
Your baseline can help indicate why your model has drifted.
To detect drift over time, set your baseline using training data to identify how your model changes between your features, predictions, and actuals. To detect short-term drift, set your baseline using historical production data (i.e. 2 weeks).
This drift indicates that the data used to build the model has shifted from the data now seen in production.
This drift indicates high variance in production data distributions, which could indicate model performance degradation, data quality issues, or noisy features.
It is common to receive delayed actuals (aka ground truth) for your production model. In this situation, use proxy metrics to evaluate model performance. The most common proxy metrics for model performance are:
Feature Drift
Prediction score/class distribution drift
Change in prediction score/class
Pro Tip: Learn how to configure the visualization of a feature's distribution comparison .
