How To Set Up Monitors
Learn how to configure your model in three ways
The Arize platform automatically detects drift, data quality issues, and anomalous performance degradations with highly configurable monitors based on both common KPIs and custom metrics.
Use this page to navigate to:
You can configure monitors in three ways:
If you're new to the platform, we suggest bulk-creating your first set of monitors to get started using Arize. These monitors provide a holistic view of your data quality, drift, and model performance. From there, customize our default monitors, or create your own monitors from scratch.
Automatically set up monitors via the 'Model Overview' page or 'Monitors' tab. From the 'Model Overview' page, scroll down to the 'Monitors' card and click on the 'Set Up Monitors' button. A pop-up will appear with the option to choose an evaluation metric and positive class for performance monitors, the option to disable a monitor type, and a drop-down menu to configure your alerts.
You can customize monitors individually from scratch or once your monitors are created from the bulk create flow. Custom monitors allow you to change your threshold, edit your evaluation metric, filter, and more.
Configure elements such as:
- Metrics: edit based on a wide range of metrics such as F_1, AUC, RMSE, and more
- Filters: filter your monitor on prediction score, feature, actual class, etc.
- Evaluation window: change the time window from 1 hour - 30 days
- Threshold value: automatic or custom, edit the multiplier within the calculated value
- Alerts: change your integration or email alerts
To create a net new custom monitor, navigate to the 'Monitors' tab and select 'New Monitor' on the top right to customize your model details.
Use our public-facing GraphQL API to bulk configure custom monitors within your own infrastructure. Follow the instructions here to query, patch, and create monitors using our programmatic API.
Arize monitors trigger an alert when your monitor crosses a threshold. Depending on your model needs, you can configure your own threshold or set an automatic threshold. Automatic thresholding is enabled by default for new monitors.
Automatic thresholds are set by Arize when there is sufficient production data to determine a trend. The threshold is determined by looking back at a historical time window for a metric and calculating the variance of data in that time period (more here). Auto thresholds are based on 14 days + a 3-day delay of historical data.
Toggle automatic thresholds on or off from the “Edit monitor” configuration.
Remember to reset old thresholds, as they do not automatically refresh from when your monitor was first configured.
Thresholds can become outdated as better or more data is collected. Sometimes a monitor fires when a threshold simply needs to be reset.
With auto thresholds turned off, set the threshold to any value. We display the mean and standard deviation values used to calculate the auto threshold. From there, change the number of standard deviations above or below the mean to calculate a suggested threshold.
Last modified 2mo ago