Overview

We have built simple functions for using our eval prompt templates. These prompts are tested against and target precision at 70-90% and F1 at 70-85%. We use , our open-source package to run evaluations.

To use our evaluators, follow these steps below.

Choose an evaluator

llm_classify runs LLM as a Judge across your LLM outputs. You can use any of the evaluation templates below. You can see notebook tutorials on how to use these in our Phoenix repo.

Have Copilot Choose an Evaluator

Using LLM_Classify

Setup the evaluation library

All of our evaluators are easily imported with the phoenix library, which you can install using this command below.

Import the pre-tested evaluators along with the helper functions using this code snippet.

Prepare your data

For this example, we will create the dataframe from scratch to include the required columns we need -- input, reference, and output.

Run the eval

Then you can use the llm_classify function to run the evals on your dataframe.

Now you have the results of your hallucination eval and QA correctness eval! See the results below when you print your results (data not included).

hallucination_eval_df

qa_correctness_eval_df

Supported Models

The models are instantiated and usable in the LLM Eval function. The models are also directly callable with strings.