Get Started With Monitors
Learn how to configure your model in three ways
Continuous monitoring ensures the accuracy and reliability of ML predictions over time. This is critical because models can drift or degrade in performance due to changes in the underlying data, altering environments, or evolving target variables.
Proactively monitor your models with Arize to automatically detect drift, data quality issues, and anomalous performance degradations with highly configurable monitors based on both common KPIs and custom metrics.
Monitoring isn't a one-size-fits-all solution for the variety of ML use cases, business needs, and areas of concern.
✌️ Two Types of Drift
Use drift monitors to compare production against different baseline datasets.
- 1.Feature drift captures changes to your data pipeline that can lead to anomalous model behavior.
- 2.Prediction drift captures changes in the outputs of your model that may require stakeholders to be notified. This is also an excellent way to monitor performance without ground truth values.
🚀 Performance
Monitor performance metrics based on ground truth data (actuals) for your model type, such as NDCG (ranking), AUC (propensity to click), MAPE (predicting ETAs), and more!
📌 Important Features
Monitor key features important to your model with data quality monitors. This can be a powerful tool for root cause analysis workflows.
🔍 Leading Indicators
If your model receives delayed ground truth, monitor your prediction drift score and feature drift as a proxy for model performance.
Start monitoring your models with Arize in one click! Pick from a wide array of different data quality, drift, and model performance metrics to monitor across all applicable features.
Enable monitors for 1 performance metric, 1 drift metric, and 1 data quality metrics to get started!
Automatically set up monitors via the 'Model Overview' page or 'Monitors' tab.
Some monitor types, such as feature drift, will set up monitors across all of your features at once. This way, you can achieve 100% coverage of drift monitoring across all of your features in one click.
All monitors are configured with an automatic threshold by default. Auto thresholds use a powerful and scalable algorithm that adjusts with your metrics to alert on anomalous behavior but not on small perturbations in the data. Auto thresholds allow you to enable a large number of monitors for wide coverage, without having to tune the threshold for each monitor.
Example Recall Monitor With An Auto Threshold
If you have a precise area to monitor or specific configuration needs, edit and customize your monitor in the UI or programmatically.
- Metrics: edit based on a wide range of metrics such as F_1, AUC, RMSE, and more
- Filters: filter your monitor on prediction score, feature, actual class, etc.
- Evaluation window: change the time window from 1 hour - 30 days
- Threshold value: automatic or custom, edit the multiplier within the calculated value
- Alerts: change your integration or email alerts
Use our public-facing GraphQL API to bulk configure custom monitors within your own infrastructure.
mutation createPerformanceMonitorForState($state: String!){
createPerformanceMonitor(
input: {
modelId: "model_id",
operator: lessThan,
performanceMetric: accuracy,
dynamicAutoThreshold: {
stdDevMultiplier: 1.2
}
filters: [{
dimensionType: featureLabel,
operator: equals
name: "stateName"
values: [$state]
}],
contacts: [{
notificationChannelType: email,
emailAddress: "[email protected]"
}]
}
) {
monitor {
id
}
}
}
Last modified 2mo ago