Quickstart: Deployment
How to use phoenix outside of the notebook environment.
Phoenix's notebook-first approach to observability makes it a great tool to utilize during experimentation and pre-production. However at some point you are going to want to ship your application and continue to monitor your application as it runs.
In order to run Phoenix tracing in production, you will have to follow these following steps:
Setup a Server: your LLM application to run on a server
Instrument: Add OpenInference Instrumentation to your server
Observe: Run the Phoenix server as or a standalone instance and point your tracing instrumentation to the phoenix server
Looking for a working application? Jump to our Python and Javascript examples.
Setup a Server
Setting up a server to run your LLM application can be tricky to bootstrap. While bootstrapping and LLM application is not part of Phoenix, you can take a look at some of examples from our partners.
create-llama: A bootstrapping tool for setting up a full-stack LlamaIndex app
langchain-templates: Create a Langchain server using a template
Note that the above scripts and templates are provided purely as examples
Instrument
In order to make your LLM application observable, it must be instrumented. That is, the code must emit traces. The instrumented data must then be sent to an Observability backend, in our case the Phoenix server.
Phoenix collects traces from your running application using OTLP (OpenTelemetry Protocol). Notably, Phoenix accepts traces produced via instrumentation provided by OpenInference. OpenInference instrumentations automatically instrument your code so that LLM Traces can be exported and collected by Phoenix. To learn more about instrumentation, check out the full details here.
OpenInference currently supports instrumenting your application in both Python and Javascript. For each of these languages, you will first need to install the opentelemetry
and openinference
packages necessary to trace your application.
Install OpenTelemetry
For a comprehensive guide to python instrumentation, please consult OpenTelemetry's guide
Install OpenTelemetry packages
Install OpenInference Instrumentations
To have your code produce LLM spans using OpenInference, you must pick the appropriate instrumentation packages and install them using a package manager. For a comprehensive list of instrumentations, checkout the OpenInference repository.
Initialize Instrumentation
In order for your application to export traces, it must be instrumented using OpenInference instrumentors. Note that instrumentation strategies differ by language so please consult OpenTelemetry's guidelines for full details.
Note that the below examples assume you are running phoenix via docker compose and thus simply have the URL http://phoenix:6006. If you are deploying phoenix separately, replace this string with the full URL of your running phoenix instance
Below is a example of what instrumentation might look like for LlamaIndex. instrument
should be called before main
is run in your server.
Observe
Lastly, we must run the phoenix server so that our application can export spans to it. To do this, we recommend running phoenix via an image. Phoenix images are available via dockerhub.
In order to run the phoenix server, you will have to start the application. Below are a few examples of how you can run the application on your local machine.
Pull the image you would like to run
Pick an image you would like to run or simply run the latest:
Note, you should pin the phoenix version for production to the version of phoenix you plan on using. E.x. arizephoenix/phoenix:4.0.0
See Portsfor details on the ports for the container.
Note that the above simply starts the phoenix server locally. A simple way to make sure your application always has a running phoenix server as a collector is to run the phoenix server as a side car.
Here is an example compose.yaml
This way you will always have a running Phoenix instance when you run
For the full details of on how to configure Phoenix, check out the Configuration section
Last updated