12.19.2022
New Releases, Enhancements, Changes + Arize in the News!
What's New
Model Type: Ranking
Ranking models are a critical component of recommender systems; native support for ranking models resolves rank-specific model issues easier and improves model outcomes faster.
Data Ingestion: Ingest model inference data across various ranking use cases to easily get started using Arize. Learn how to ingest ranking model data here.
Rank-aware Evaluation Metrics: Compute model performance using metrics that gauge rank order and relevancy of predictions. Arize currently supports NDCG @ k (AUC, MRR, MAP @ k coming soon).
Check out the arize-hotel-demo-ranking model in the product quickstart guide for a tutorial on troubleshooting model performance using NDCG @ k.
Custom Metrics
Custom metrics enable you to define any metric using a combination of existing model dimensions and functions. Written using a SQL-like query language, create custom metrics by combining your model's inference data (features, predictions, actuals, etc.) using aggregations, operators, and metric functions. Use custom metrics across the platform in dashboards and monitors to share with stakeholders and receive alerts when your metric deviates from expected values.
Learn how to create custom metrics here.
Enhancements
Color by Tag
Color the UMAP point cloud by tag and view details to uncover more information about your embedding drift.
In The News
Learn how to use custom metrics to automate AI ROI calculations, map impact across all model inference data, and reduce overall costs.
We're thrilled to announce our partnership with BentoML, which streamlines the machine learning development lifecycle to supercharge production ML. This integration enables users to create, ship, troubleshoot, and improve models in real time. Learn more about how Arize AI and BentoML help continuously ship new models and improve model performance in our latest blog, co-published with BentoML
'What Is' Series:
Learn how to calculate precision, a common performance metric, and when to use it
A fast and intuitive explanation of calibration and reliability curve
A practical guide on how to calculate KL divergence, the intuition behind the metric, and how to best use it in model monitoring
Approaches for measuring embedding/vector drift for unstructured data — including for computer vision and natural language processing models.
Last updated
