Phoenix
Ask or search…
K
Links

Deploying Phoenix

How to use phoenix outside of the notebook environment.
The phoenix server can be run as a collector of spans over OTLP
Phoenix's notebook-first approach to observability makes it a great tool to utilize during experimentation and pre-production. However at some point you are going to want to ship your application to production and continue to monitor your application as it runs.
Phoenix is made up of two components that can be deployed independently:
  • Trace Instrumentation: These are a set of plugins that can be added to your application's startup process. These plugins (known as instrumentations) automatically collect spans for your application and export them for collection and visualization. For phoenix, all the instrumentors are managed via a single repository called OpenInference
  • Trace Collector: The Phoenix server acts as a trace collector and application that helps you troubleshoot your application in real time.
In order to run Phoenix tracing in production, you will have to follow these following steps:
  1. 1.
    Setup a Server: your LLM application to run on a server
  2. 2.
    Instrument: Add OpenInference Instrumentation to your server
  3. 3.
    Observe: Run the Phoenix server as a side-car or a standalone instance and point your tracing instrumentation to the phoenix server

Working Examples

Below are example repositories of how to setup an LLM application in Python and Javascript

Setup a Server

Setting up a server to run your LLM application can be tricky to bootstrap. While bootstrapping and LLM application is not part of Phoenix, you can take a look at some of examples from our partners.
Note that the above scripts and templates are provided purely as examples

Instrument

In order to make your LLM application observable, it must be instrumented. That is, the code must emit traces. The instrumented data must then be sent to an Observability backend, in our case the Phoenix server.
Phoenix collects traces from your running application using OTLP (OpenTelemetry Protocol). Notably, Phoenix accepts traces produced via instrumentation provided by OpenInference. OpenInference instrumentations automatically instrument your code so that LLM Traces can be exported and collected by Phoenix. To learn more about instrumentation, check out the full details here.
OpenInference currently supports instrumenting your application in both Python and Javascript. For each of these languages, you will first need to install the opentelemetry and openinference packages necessary to trace your application.

Install OpenTelemetry

Python Dependancies
Javascript Dependancies
For a comprehensive guide to python instrumentation, please consult OpenTelemetry's guide

Install OpenTelemetry packages

pip install opentelemetry-api opentelemetry-instrumentation opentelemetry-semantic-conventions opentelemetry-exporter-otlp-proto-http
For a comprehensive guide on instrumenting NodeJS using OpenTelemetry, consult their guide
npm install @opentelemetry/exporter-trace-otlp-proto @opentelemetry/resources @opentelemetry/sdk-trace-node --save

Install OpenInference Instrumentations

To have your code produce LLM spans using OpenInference, you must pick the appropriate instrumentation packages and install them using a package manager. For a comprehensive list of instrumentations, checkout the OpenInference repository.

Initialize Instrumentation

In order for your application to export traces, it must be instrumented using OpenInference instrumentors. Note that instrumentation strategies differ by language so please consult OpenTelemetry's guidelines for full details.
Note that the below examples assume you are running phoenix via docker compose and thus simply have the URL http://phoenix:6006. If you are deploying phoenix separately, replace this string with the full URL of your running phoenix instance
Python
Javascript
Below is a example of what instrumentation might look like for LlamaIndex. instrument should be called before main is run in your server.
from openinference.instrumentation.llama_index import LlamaIndexInstrumentor
from opentelemetry import trace as trace_api
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk import trace as trace_sdk
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
def instrument():
tracer_provider = trace_sdk.TracerProvider()
span_exporter = OTLPSpanExporter("http://phoenix:6006/v1/traces")
span_processor = SimpleSpanProcessor(span_exporter)
tracer_provider.add_span_processor(span_processor)
trace_api.set_tracer_provider(tracer_provider)
LlamaIndexInstrumentor().instrument()
Code below is written in ESM format
For instrumentation to work with NodeJS to work, you must create a file instrumentation.js and have it run BEFORE all other server code in index.js
place the following code in a instrumentation.js file
import { registerInstrumentations } from "@opentelemetry/instrumentation";
import { OpenAIInstrumentation } from "@arizeai/openinference-instrumentation-openai";
import {
ConsoleSpanExporter,
SimpleSpanProcessor,
} from "@opentelemetry/sdk-trace-base";
import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node";
import { Resource } from "@opentelemetry/resources";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";
import { SemanticResourceAttributes } from "@opentelemetry/semantic-conventions";
import { diag, DiagConsoleLogger, DiagLogLevel } from "@opentelemetry/api";
// For troubleshooting, set the log level to DiagLogLevel.DEBUG
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG);
const provider = new NodeTracerProvider({
resource: new Resource({
[SemanticResourceAttributes.SERVICE_NAME]: "openai-service",
}),
});
provider.addSpanProcessor(new SimpleSpanProcessor(new ConsoleSpanExporter()));
provider.addSpanProcessor(
new SimpleSpanProcessor(
new OTLPTraceExporter({
url: "http://localhost:6006/v1/traces",
}),
),
);
provider.register();
registerInstrumentations({
instrumentations: [new OpenAIInstrumentation({})],
});
console.log("👀 OpenInference initialized");// Some code
Then make sure that this file is required before running the server.
node -r instrumentation.js index.js

Observe

Lastly, we must run the phoenix server so that our application can export spans to it. To do this, we recommend running phoenix via an image. Phoenix images are available via dockerhub.
In order to run the phoenix server, you will have to start the application. Below are a few examples of how you can run the application on your local machine.
Docker
Command Line
Pull the image you would like to run
docker pull arizephoenix/phoenix
Pick an image you would like to run or simply run the latest:
Note, you should pin the phoenix version for production to the version of phoenix you plan on using. E.x. arizephoenix/phoenix:2.7.0
docker run -p 6006:6006 -i -t arizephoenix/phoenix:latest
python3 -m phoenix.server.main --port 6006 serve
Note that the above simply starts the phoenix server locally. A simple way to make sure your application always has a running phoenix server as a collector is to run the phoenix server as a side car.
here is a n example compose.yaml
services:
phoenix:
image: arizephoenix/phoenix:latest
ports:
- "6006:6006"
backend:
build:
context: ./backend
dockerfile: Dockerfile
args:
OPENAI_API_KEY: ${OPENAI_API_KEY}
ports:
- "8000:8000"
environment:
- OPENAI_API_KEY=${OPENAI_API_KEY}
- COLLECTOR_ENDPOINT=http://phoenix:6006/v1/traces
- PROD_CORS_ORIGIN=http://localhost:3000
# Set INSTRUMENT_LLAMA_INDEX=false to disable instrumentation
- INSTRUMENT_LLAMA_INDEX=true
healthcheck:
test: ["CMD", "wget", "--spider", "http://0.0.0.0:8000/api/chat/healthcheck"]
interval: 5s
timeout: 1s
retries: 5
frontend:
build: frontend
ports:
- "3000:3000"
depends_on:
backend:
condition: service_healthy
This way you will always have a running Phoenix instance when you run
docker compose up