Object Detection

How to declare your model schema and log data for object detection models

Object Detection Model Overview

Object detection models identify and locate objects within images or videos by assigning them specific bounding boxes.

Applicable Metrics: Accuracy, Euclidian Distance (embeddings)

Click here for all valid model types and metric combinations.

Object Detection Code Example

Example Row

image_vector
image_link
prediction_bboxes
actual_bboxes
prediction_categories
actual_categories
actual_super_categories
prediction_scores
Timestamp

[0.24713118374347687, 0.7061651349067688, 1.12...

"https://link-to-my-image.png"

[[50.43, 109.49, 538.21...

[[55.39, 107.72, 539.25, 362.9], [554.41, 194....

[bus]

[bus, person, person]

[vehicle, person, person]

[0.9997552]

1618590882
embedding_feature_column_names={
    "image_embedding": EmbeddingColumnNames(
        vector_column_name="image_vector",
        link_to_data_column_name="url"
    )
}
object_detection_prediction_column_names=ObjectDetectionColumnNames(
    bounding_boxes_coordinates_column_name="prediction_bboxes",
    categories_column_name="prediction_categories",
    scores_column_name="prediction_scores"
)
object_detection_actual_column_names=ObjectDetectionColumnNames(
    bounding_boxes_coordinates_column_name="actual_bboxes",
    categories_column_name="actual_categories",
)

# feature & tag columns can be optionally defined with typing:
tag_columns = TypedColumns(
    inferred=["name"],
    to_int=["zip_code", "age"]
)

# Defina the Schema, including embedding information
schema = Schema(
    prediction_id_column_name="prediction_id",
    timestamp_column_name="prediction_ts",
    tag_column_names=tag_columns,
    embedding_feature_column_names=embedding_feature_column_names,
    object_detection_prediction_column_names=object_detection_prediction_column_names,
    object_detection_actual_column_names=object_detection_actual_column_names,
)

response = arize_client.log(
    model_id= "CV-object-detection",
    model_version= "v1",
    model_type=ModelTypes.OBJECT_DETECTION,
    environment=Environments.PRODUCTION,
    dataframe=df,
    schema=schema
)

For more details on Python Batch API Reference, visit here:

Pandas Batch Logging

Object Detection Prediction & Actual Values

Arize supports logging object detection prediction and actual values using the ObjectDetectionColumnNames object, which can be assigned to the prediction/actual schema parameters, object_detection_prediction_column_names and object_detection_actual_column_names.

Object prediction or actual declaration is required to use the object detection model type in Arize.

class ObjectDetectionColumnNames(
    bounding_boxes_coordinates_column_name: str
    categories_column_name: str
    scores_column_name: Optional[str] = None # actual ground truth labels wont have scores
)

Embedding Features

In addition to object detection prediction and actual values, Arize supports logging the embedding features associated with the images in an object detection model using the EmbeddingColumnNames object.

  • The vector_column_name should be the name of the column where the embedding vectors are stored. The embedding vector is the dense vector representation of the unstructured input. ⚠️ Note: embedding features are not sparse vectors.

  • The link_to_data_column_name should be the name of the column where the URL links to the source images are stored.

{ 
    "embedding_display_name": EmbeddingColumnNames(
        vector_column_name="image_vector", 
        link_to_data_column_name="image_link" 
    ) 
}

See here for more information on embeddings and options for generating them.

Last updated

Was this helpful?