Search
K
Links

Object Detection

How to declare your model schema and log data for object detection models
Object detection model type support is in early release. Reach out to [email protected] for access.

Object Detection Model Overview

Object detection models identify and locate objects within images or videos by assigning them specific bounding boxes.
Applicable Metrics: Accuracy, Euclidian Distance (embeddings)
Click here for all valid model types and metric combinations.

Object Detection Code Example

Python Batch

Example Row

image_vector
image_link
prediction_bboxes
actual_bboxes
prediction_categories
actual_categories
actual_super_categories
prediction_scores
Timestamp
[0.24713118374347687, 0.7061651349067688, 1.12...
"https://link-to-my-image.png"
[[50.43, 109.49, 538.21...
[[55.39, 107.72, 539.25, 362.9], [554.41, 194....
[bus]
[bus, person, person]
[vehicle, person, person]
[0.9997552]
1618590882
embedding_feature_column_names={
"image_embedding": EmbeddingColumnNames(
vector_column_name="image_vector",
link_to_data_column_name="url"
)
}
object_detection_prediction_column_names=ObjectDetectionColumnNames(
bounding_boxes_coordinates_column_name="prediction_bboxes",
categories_column_name="prediction_categories",
scores_column_name="prediction_scores"
)
object_detection_actual_column_names=ObjectDetectionColumnNames(
bounding_boxes_coordinates_column_name="actual_bboxes",
categories_column_name="actual_categories",
)
# Defina the Schema, including embedding information
schema = Schema(
prediction_id_column_name="prediction_id",
timestamp_column_name="prediction_ts",
tag_column_names=tags,
embedding_feature_column_names=embedding_feature_column_names,
object_detection_prediction_column_names=object_detection_prediction_column_names,
object_detection_actual_column_names=object_detection_actual_column_names,
)
response = arize_client.log(
model_id= "CV-object-detection",
model_version= "v1",
model_type=ModelTypes.OBJECT_DETECTION,
environment=Environments.PRODUCTION,
dataframe=df,
schema=schema
)
For more details on Python Batch API Reference, visit here:

Object Detection Prediction & Actual Values

Arize supports logging object detection prediction and actual values using the ObjectDetectionColumnNames object, which can be assigned to the prediction/actual schema parameters, object_detection_prediction_column_names and object_detection_actual_column_names.
Object prediction or actual declaration is required to use the object detection model type in Arize.
class ObjectDetectionColumnNames(
bounding_boxes_coordinates_column_name: str
categories_column_name: str
scores_column_name: Optional[str] = None # actual ground truth labels wont have scores
)

Embedding Features

In addition to object detection prediction and actual values, Arize supports logging the embedding features associated with the images in an object detection model using the EmbeddingColumnNames object.
  • The vector_column_name should be the name of the column where the embedding vectors are stored. The embedding vector is the dense vector representation of the unstructured input.
    Note: embedding features are not sparse vectors.
  • The link_to_data_column_name should be the name of the column where the URL links to the source images are stored.
{
"embedding_display_name": EmbeddingColumnNames(
vector_column_name="image_vector",
link_to_data_column_name="image_link"
)
}
See here for more information on embeddings and options for generating them.