User Guide: ML

Resources for Best Practices in ML Observability

ML Observability is the practice of obtaining a deep understanding into your model’s data and performance across its lifecycle. Observability doesn't just stop at surfacing a red or green light, but enables ML practitioners to root cause/explain why a model is behaving a certain way in order to improve it. Check out how Arize works across the ML Lifecycle to get the most out of ML Observability.

Arize across the ML Workflow

ML Observability: Resources

• ML Observability: Industry Certification
• ML Observability: Advanced Course
• ML Observability 101 Intro Video
• ML Observability 101: Ebook
• Model Performance Management (Paper)
• What To Look for In An ML Observability Platform (Buyer's Checklist)
• A Guide To Automated Model Retraining
• Central ML: Best Practices for Ramping Up on ML Observability

ML observability in context

ML Observability: Fundamentals

What Is Observability?

• ML Observability: The Essentials
• Tracing In Machine Learning

Model Evaluation Metrics

• Binary Cross Entropy
• Precision
• Recall
• F1 Score
• Calibration Curve
• PR AUC
• AUC ROC
• Mean Absolute Percentage Error (MAPE)
• Normalized Discounted Cumulative Gain (NDCG)
• Other Rank Aware Evaluation Metrics

Drift Metrics

• Data Binning
• Population Stability Index (PSI)
• KL Divergence
• Jensen Shannon Divergence
• Kolmogorov Smirnov Test

Fairness & Bias Metrics

• Bias Tracing

Data Quality

• Solving Data Quality With ML Observability

Service Monitoring Metrics

• ML Service-Level Performance Monitoring Essentials

Explainability

• Explainability Techniques

Monitoring Image and Language Models and Embeddings

• KNN Algorithm
• Tokenization
• Embedding Versioning
• Dimensionality Reduction
• Monitoring Embedding/Vector Drift
• BERT
• Bleu Score and Other Large Language Model Metrics