Generative Skills

AI Search

• Suggested Prompt: "Search <col_name> for <search_criteria>"

• Use When: You want to search your LLM data using natural language queries.

• Description: Searches your LLM data using natural language queries. Copilot identifies relevant traces for further analysis.

Pro tip: Use AI Search right from the filter bar

Create Custom Evaluations

• Suggested Prompt: "Write a custom evaluation that meets the following criteria:"

• Use When: You need a tailored eval for your application.

• Description: Writes a tailored eval for your application. Specify your goal or let Copilot analyze your data and make suggestions.

Pro tip: Create Custom Evaluation with the mini chat in the Task Creation Modal

Diagnose RAG Issues

• Suggested Prompt: "Debug retrieval step of RAG app for <trace_id>"

• Use When: You need to refine your retrieval process.

• Description: Evaluate and refine your retrieval process. Analyzes responses to ensure relevance and accuracy, and provides improvement strategies.

Pro tip: Debug retrieval directly from trace details

Optimize Prompts

• Suggested Prompt: "Optimize my prompt to <goal>"

• Use When: You want to fine-tune your prompts.

• Description: Optimize your prompt to address specific issues or enhance overall response quality.

Pro tip: Optimize and then test prompt templates in the prompt playground

Summarize Evaluation Metrics

• Suggested Prompt: "Summarize my <eval_name> eval"

• Use When: You want to understand the performance of your LLM application.

• Description: Assesses and summarizes your evaluation metrics, offering targeted suggestions for enhancing your LLM application.