Model Performance

Model Performance Templates

Model performance templates help you track the health of models with actuals data.

Regression Model

Concisely overview aspects that impact regression model performance

Key Widgets: Count statistics, error measures (RMSE, MAE, MSE, MAPE), average predictions, daily trends, and feature distributions, prediction vs. actual comparison charts

Classification Model

Concisely overview aspects that impact classification model performance

Key Widgets: Statistics (prediction/actual counts, accuracy, score averages), time series charts (class comparisons, daily values, score distributions), and specialized charts for accuracy, F1 scores by class, and feature distribution

Ranking Model

Concisely overview aspects that impact ranking model performance

Key Widgets: Rank performance metrics (NDCG, MRR, AUC, MAP) for overall model performance and individual features, model performance based on rank groups, and time-series fluctuations

Compare A/B

Compare two models in production using performance metrics

Key Widgets: Compare two models based on performance and data quality metrics over time, implement canary vs live model comparison, and compare & track significant production data fluctuations.

Production vs Training

Compare and visualize the difference between production performance to training performance

Key Widgets: Track slices and compare between production and training environments. This template creates an aggregate count tracker for key data and an overlaid view of metric history over time