Generate Embeddings

What is an Embedding?

Embeddings are vector representations of data. Embeddings are everywhere in modern deep learning, such as transformers, recommendation engines, layers of deep neural networks, encoders, and decoders.

Why Embeddings for Analyzing Deep Learning Models?

Data drift in unstructured data like images is complicated to measure. The measures typically used for drift in structured data do not extend to unstructured data. The general challenge with measuring unstructured data drift is that you need to understand the change in relationships inside the unstructured data itself.

How to Generate Embeddings

Arize supports 2 approaches - bring your own embeddings or have Arize generate them for you.

Bring/Generate Your Own Embedding

Let Arize Generate Your Embeddings

Why are Embeddings So Important?

Embeddings are foundational because:

1. They can represent images, audio signals, and even large chunks of structured data.

2. They provide a common mathematical representation of your data

3. They compress your data

4. They preserve relationships within your data

5. They are the output of deep learning layers providing comprehensible linear views into complex non-linear relationships learned by models

Example of Image, image embedding, and the embedding projection to 2-D

Additional Resources

Check out our tutorials on how to send embeddings to Arize for different use cases.

Getting Started: Quick Guides	Category	Code
Multi-Class Sentiment Classification	NLP	Colab Link
Named Entity Recognition	NLP	Colab Link
Image Classification	CV	Colab Link

Learn more about embeddings and troubleshooting with Arize:

• Blog: Monitor Unstructured Data with Arize

• Blog: Getting Started With Embeddings is Easier Than You Think

• Talk: Embedding Usage and Visualization In Modern ML Systems

Questions? Email us at [e-mail] or Slack us in the #arize-support channel