Last updated
Copyright © 2023 Arize AI, Inc
Last updated
There are subtle differences between the experiments SDK using Arize vs. Phoenix, but the base are the same. The example below runs an experiment to write a haiku, and evaluate its tone using an .
You can check out a full notebook example of each.
Arize uses the ArizeDatasetsClient
, which requires a developer key you can find in the Arize product. The arize_client.create_dataset
function also returns a dataset_id, instead of a dataset object. So if you want to print or manipulate the dataset, you will need to get the dataset using arize_client.get_dataset
.
You can find your developer key under the GraphQL on the bottom left hand side nav -> Get your developer key. Note: You will need developer access under Your Settings -> Manage Account -> Developer Access
Phoenix uses px.Client().upload_dataset
.
We define the LLM call here, which uses data from the dataset_row as prompt template variables in Arize. In Phoenix, we use the input variable to capture the items in the dataset.
For both Arize and Phoenix, you can often use the exact same function as your evaluator. Phoenix does have slightly different way of accessing metadata from your dataset.
Arize and Phoenix uses slightly different functions to run an experiment. Arize requires the space_id to be passed in, where Phoenix does not have spaces.
In Arize, you also pass in the dataset_id, instead of the dataset object itself in Phoenix.
uses input, output, dataset_row, metadata
as the optional input variables to pass into the function.
uses input, expected, reference, example, metadata
as the input variables to pass into the function.