LogoLogo
Python SDKSlack
  • Documentation
  • Cookbooks
  • Self-Hosting
  • Release Notes
  • Reference
  • Arize AI
  • Quickstarts
  • โœจArize Copilot
  • Arize AI for Agents
  • Concepts
    • Agent Evaluation
    • Tracing
      • What is OpenTelemetry?
      • What is OpenInference?
      • Openinference Semantic Conventions
    • Evaluation
  • ๐ŸงชDevelop
    • Quickstart: Experiments
    • Datasets
      • Create a dataset
      • Update a dataset
      • Export a dataset
    • Experiments
      • Run experiments
      • Run experiments with code
        • Experiments SDK differences in AX vs Phoenix
        • Log experiment results via SDK
      • Evaluate experiments
      • Evaluate experiment with code
      • CI/CD with experiments
        • Github Action Basics
        • Gitlab CI/CD Basics
      • Download experiment
    • Prompt Playground
      • Use tool calling
      • Use image inputs
      • Replay spans
      • Compare prompts side-by-side
      • Load a dataset into playground
      • Save playground outputs as an experiment
      • โœจCopilot: prompt builder
    • Playground Integrations
      • OpenAI
      • Azure OpenAI
      • AWS Bedrock
      • VertexAI
      • Custom LLM Models
    • Prompt Hub
  • ๐Ÿง Evaluate
    • Online Evals
      • Run evaluations in the UI
      • Run evaluations with code
      • Test LLM evaluator in playground
      • View task details & logs
      • โœจCopilot: Eval Builder
      • โœจCopilot: Eval Analysis
      • โœจCopilot: RAG Analysis
    • Experiment Evals
    • LLM as a Judge
      • Custom Eval Templates
      • Arize Templates
        • Agent Tool Calling
        • Agent Tool Selection
        • Agent Parameter Extraction
        • Agent Path Convergence
        • Agent Planning
        • Agent Reflection
        • Hallucinations
        • Q&A on Retrieved Data
        • Summarization
        • Code Generation
        • Toxicity
        • AI vs Human (Groundtruth)
        • Citation
        • User Frustration
        • SQL Generation
    • Code Evaluations
    • Human Annotations
  • ๐Ÿ”ญObserve
    • Quickstart: Tracing
    • Tracing
      • Setup tracing
      • Trace manually
        • Trace inputs and outputs
        • Trace function calls
        • Trace LLM, Retriever and Tool Spans
        • Trace prompt templates & variables
        • Trace as Inferences
        • Send Traces from Phoenix -> Arize
        • Advanced Tracing (OTEL) Examples
      • Add metadata
        • Add events, exceptions and status
        • Logging Latent Metadata
        • Add attributes, metadata and tags
        • Send data to a specific project
        • Get the current span context and tracer
      • Configure tracing options
        • Configure OTEL tracer
        • Mask span attributes
        • Redact sensitive data from traces
        • Instrument with OpenInference helpers
      • Query traces
        • Filter Traces
          • Time Filtering
        • Export Traces
        • โœจAI Powered Search & Filter
        • โœจAI Powered Trace Analysis
        • โœจAI Span Analysis & Evaluation
    • Tracing Integrations
      • OpenAI
      • OpenAI Agents SDK
      • LlamaIndex
      • LlamaIndex Workflows
      • LangChain
      • LangGraph
      • Hugging Face smolagents
      • Autogen
      • Google GenAI (Gemini)
      • Model Context Protocol (MCP)
      • Vertex AI
      • Amazon Bedrock
      • Amazon Bedrock Agents
      • MistralAI
      • Anthropic
      • LangFlow
      • Haystack
      • LiteLLM
      • CrewAI
      • Groq
      • DSPy
      • Guardrails AI
      • Prompt flow
      • Vercel AI SDK
      • Llama
      • Together AI
      • OpenTelemetry (arize-otel)
      • BeeAI
    • Evals on Traces
    • Guardrails
    • Sessions
    • Dashboards
      • Dashboard Widgets
      • Tracking Token Usage
      • โœจCopilot: Dashboard Widget Creation
    • Monitors
      • Integrations: Monitors
        • Slack
          • Manual Setup
        • OpsGenie
        • PagerDuty
      • LLM Red Teaming
    • Custom Metrics & Analytics
      • Arize Query Language Syntax
        • Conditionals and Filters
        • All Operators
        • All Functions
      • Custom Metric Examples
      • โœจCopilot: ArizeQL Generator
  • ๐Ÿ“ˆMachine Learning
    • Machine Learning
      • User Guide: ML
      • Quickstart: ML
      • Concepts: ML
        • What Is A Model Schema
        • Delayed Actuals and Tags
        • ML Glossary
      • How To: ML
        • Upload Data to Arize
          • Pandas SDK Example
          • Local File Upload
            • File Upload FAQ
          • Table Ingestion Tuning
          • Wildcard Paths for Cloud Storage
          • Troubleshoot Data Upload
          • Sending Data FAQ
        • Monitors
          • ML Monitor Types
          • Configure Monitors
            • Notifications Providers
          • Programmatically Create Monitors
          • Best Practices for Monitors
        • Dashboards
          • Dashboard Widgets
          • Dashboard Templates
            • Model Performance
            • Pre-Production Performance
            • Feature Analysis
            • Drift
          • Programmatically Create Dashboards
        • Performance Tracing
          • Time Filtering
          • โœจCopilot: Performance Insights
        • Drift Tracing
          • โœจCopilot: Drift Insights
          • Data Distribution Visualization
          • Embeddings for Tabular Data (Multivariate Drift)
        • Custom Metrics
          • Arize Query Language Syntax
            • Conditionals and Filters
            • All Operators
            • All Functions
          • Custom Metric Examples
          • Custom Metrics Query Language
          • โœจCopilot: ArizeQL Generator
        • Troubleshoot Data Quality
          • โœจCopilot: Data Quality Insights
        • Explainability
          • Interpreting & Analyzing Feature Importance Values
          • SHAP
          • Surrogate Model
          • Explainability FAQ
          • Model Explainability
        • Bias Tracing (Fairness)
        • Export Data to Notebook
        • Automate Model Retraining
        • ML FAQ
      • Use Cases: ML
        • Binary Classification
          • Fraud
          • Insurance
        • Multi-Class Classification
        • Regression
          • Lending
          • Customer Lifetime Value
          • Click-Through Rate
        • Timeseries Forecasting
          • Demand Forecasting
          • Churn Forecasting
        • Ranking
          • Collaborative Filtering
          • Search Ranking
        • Natural Language Processing (NLP)
        • Common Industry Use Cases
      • Integrations: ML
        • Google BigQuery
          • GBQ Views
          • Google BigQuery FAQ
        • Snowflake
          • Snowflake Permissions Configuration
        • Databricks
        • Google Cloud Storage (GCS)
        • Azure Blob Storage
        • AWS S3
          • Private Image Link Access Via AWS S3
        • Kafka
        • Airflow Retrain
        • Amazon EventBridge Retrain
        • MLOps Partners
          • Algorithmia
          • Anyscale
          • Azure & Databricks
          • BentoML
          • CML (DVC)
          • Deepnote
          • Feast
          • Google Cloud ML
          • Hugging Face
          • LangChain ๐Ÿฆœ๐Ÿ”—
          • MLflow
          • Neptune
          • Paperspace
          • PySpark
          • Ray Serve (Anyscale)
          • SageMaker
            • Batch
            • RealTime
            • Notebook Instance with Greater than 20GB of Data
          • Spell
          • UbiOps
          • Weights & Biases
      • API Reference: ML
        • Python SDK
          • Pandas Batch Logging
            • Client
            • log
            • Schema
            • TypedColumns
            • EmbeddingColumnNames
            • ObjectDetectionColumnNames
            • PromptTemplateColumnNames
            • LLMConfigColumnNames
            • LLMRunMetadataColumnNames
            • NLP_Metrics
            • AutoEmbeddings
            • utils.types.ModelTypes
            • utils.types.Metrics
            • utils.types.Environments
          • Single Record Logging
            • Client
            • log
            • TypedValue
            • Ranking
            • Multi-Class
            • Object Detection
            • Embedding
            • LLMRunMetadata
            • utils.types.ModelTypes
            • utils.types.Metrics
            • utils.types.Environments
        • Java SDK
          • Constructor
          • log
          • bulkLog
          • logValidationRecords
          • logTrainingRecords
        • R SDK
          • Client$new()
          • Client$log()
        • Rest API
    • Computer Vision
      • How to: CV
        • Generate Embeddings
          • How to Generate Your Own Embedding
          • Let Arize Generate Your Embeddings
        • Embedding & Cluster Analyzer
        • โœจCopilot: Embedding Summarization
        • Similarity Search
        • Embedding Drift
        • Embeddings FAQ
      • Integrations: CV
      • Use Cases: CV
        • Image Classification
        • Image Segmentation
        • Object Detection
      • API Reference: CV
Powered by GitBook

Support

  • Chat Us On Slack
  • support@arize.com

Get Started

  • Signup For Free
  • Book A Demo

Copyright ยฉ 2025 Arize AI, Inc

On this page
  • AI-Powered Skills
  • Skills for LLM Applications
  • Skills for ML & CV Models
  • Data Privacy
  • Third-Party Integrations

Was this helpful?

Arize Copilot

Use AI to build and troubleshoot AI

Last updated 1 month ago

Was this helpful?

Arize Copilot is an AI assistant to help AI engineers build and improve their applications. Copilot has support for over 30+ skills including prompt optimizer, eval builder, AI search, etc. It is integrated into the product and also available as a Chat Experience.

AI-Powered Skills

Copilot has 30+ skills across both generative applications and tabular/CV use cases.

Skills for LLM Applications

Skill

Description

Searches a specific column based on the userโ€™s input to find relevant data. Example: "Find me confused inputs."

Searches across the entire table to identify patterns, anomalies, or outliers. Example: "Find inputs that reference pricing that are hallucinated."

Generates query filters based on natural language commands. Example: "Filter by input contains SDK."

Provides suggestions for search results and finds patterns in the data. Example: "What are the top 5 types of questions asked?"

Writes a tailored eval for your application based on specified goals or data analysis.

Analyzes responses in the retrieval process, ensuring relevance and accuracy, offering improvements.

Optimizes prompts to enhance response quality or address specific issues.

Assesses and summarizes evaluation metrics, providing suggestions for enhancing performance.

Generates custom metrics by translating natural language descriptions or existing code (e.g., SQL, Python) into AQL for easy application.

Generates dashboard widgets by translating natural language descriptions or existing code.

Enables effortless analysis and evaluation of spans through natural-language interactions, providing insights, answering questions, and running evaluations with ease.

Skills for ML & CV Models

Skill

Description

Provides a high-level analysis of model performance, including trends over time, prediction volumes, and drift. Best for general inquiries, not suited for detailed debugging.

Analyzes model performance across different cohorts or slices of data, identifying poorly performing segments. Provides insights into behavior over a specific period.

Pinpoints sudden input quality issues by examining features and tags for drift. Compares current distributions to a baseline to detect significant shifts.

Analyzes input data to report the percentage of missing data in features and tags, highlighting any sudden spikes or changes that could impact model inputs.

Assists in debugging issues by analyzing dataset metrics and focusing on specific dimensions. Identifies critical changes and provides actionable suggestions.

Analyzes a dimensionโ€™s distribution to understand shifts in percentage over time.

Analyzes changes in the cardinality of features and tags over time, highlighting unusual variations that may indicate data quality issues.

Provides concise summaries of embedding data, helping you quickly understand patterns and insights from your models' embeddings

Generates custom metrics by translating natural language descriptions or existing code (e.g., SQL, Python) into AQL for easy application.

Generates dashboard widgets by translating natural language descriptions or existing code.

Data Privacy

Arize Copilot is built on Azure OpenAI because of its built-in security and compliance features. This ensures that customer data is protected and not directly exposed to third-party providers. Here's how it works:

  • Data Processing: Azure acts as the data processor for the prompts and outputs sent to and generated by Copilot. The models are stateless, meaning no prompts or outputs are stored in the model.

  • No Data Sharing or Model Improvement:

    • Inputs and outputs of Copilot are NOT used to improve OpenAI models.

    • They are NOT used to improve any Microsoft or third-party products or services.

    • They are NOT used for automatically improving Azure OpenAI models.

  • Full Control by Microsoft: The Azure OpenAI Service is fully controlled by Microsoft, and the OpenAI models are hosted in Microsoftโ€™s Azure environment. The service does NOT interact with any other OpenAI-operated services, such as ChatGPT or the OpenAI API.

  • Security & Compliance: Azure OpenAI ensures that we meet industry-standard security and compliance measures, protecting your data throughout the process.

For a detailed breakdown of the data flow and additional privacy measures, refer to the diagram below:

Third-Party Integrations

Copilot includes a support skill designed to help answer user questions. When you ask a support-related question, the question itself is sent to a third-party service, RunLLM, for processing. It's important to note that:

  • Limited Data Sharing: Only the specific question you ask is shared with RunLLM. No additional model information or user data is shared beyond the question itself.

  • User Control: You retain control over your interaction with this skill. If at any time you wish to modify or revoke your consent to share support questions with RunLLM, please contact us at support@arize.com

  • Disclaimer Acknowledgement: Before using the support skill, users must acknowledge a one-time disclaimer outlining the involvement of RunLLM.

To read more about Azure Data privacy, see their documentation . If you have further questions or need more clarification on how your data is managed, feel free to contact our team at support@arize.com.

โœจ
here
AI Search - LLM Analysis Lite
Create Custom Evaluations
Diagnose RAG Issues
Optimize Prompts
Summarize Evaluation Metrics
ArizeQL Generator
Dashboard Generator
Span Chat
Get Model Insights
Cohort Performance Analysis
Detect Data Drift
Check for Missing Data
Assess Feature Data Quality
Evaluate Distribution Shifts
Review Cardinality Trends
Embedding Summarization
ArizeQL Generator
Dashboard Generator
AI Search - Column Search
AI Search - Table Search
AI Search - Text to Filter