Arize AI
Search…
File Importer
Cloud Storage
Check out our Cloud Storage Integration Documentation to get started!
Connect a cloud storage bucket with Arize to automatically sync your model inferences.
If you were already storing your model inferences in cloud storage, Arize can automatically extract data from your files and transcribe the stored rows as Arize model records.
The following tutorial walks through importing a single file into Arize using the file importer. This example highlights Amazon S3 storage.
Click file imports and choose the cloud storage option based on your cloud provider.

1. Signup for Arize and click on File Importer

The file importer is the upload icon in the left navigation.

2. Create new file import job

The file import job represents importing a new model into the Arize platform. The job will constantly check for new files that are placed in the pointed to bucket (or subdirectories) and import those files.
Only new files are imported, old files that are edited are not re-imported.
A detailed description of the file import service can be found: File Importer - Cloud Storage
File Import Job Section
Create a new file import job by clicking new import job and select Amazon S3.

3. Choose an S3 bucket and apply policy from platform

The file import job setup will walk through how to setup and apply a policy to an S3 bucket.
Bucket Setup Example
Once the bucket and prefix are setup you can drop a test file into the prefix directory:
test_data_quickstart.csv
13KB
Text
The file above has a prediction_ts column that fixes the time of the predictions to April 16th 2022. The date picker must be in this range to see the data in the UI. You can also edit the prediction_ts field to more recent values: https://www.epochconverter.com/
The bucket policy should be copied and placed on permissions of the S3 bucket.
Example of S3 User Interface
If configuring S3 bucket using Amazon UI go to the specific buckets permissions section and paste into Bucket Policy section.

4. Model Configuration

The model configuration defines the needed model metadata:
  • Model ID
  • Model Type
  • Environment
    • Production
    • Training
    • Validation
In this example we are going to setup:
  • Model ID = test-fileimporter-model
  • Model Type = Score
  • Environment = Production

5. Configure Schema

The following shows the default schema mapping for the file importer.
Please update the default schema with the following, so that it will match the test file provided above. Full details on this schema are available in File Schema.
{
"prediction_id": "prediction_id",
"prediction_label": "prediction_label",
"timestamp": "prediction_ts",
"actual_label": "actual_label",
"version": "",
"prediction_score": "prediction_score",
"actual_score": "actual_score",
"tags": "tag/",
"shap_values": "shap/",
"exclude": []
}

6. Finish

The file should be imported within 15 minutes of setting up the importer job. You can check the status on the import page.
Copy link
Outline
1. Signup for Arize and click on File Importer
2. Create new file import job
3. Choose an S3 bucket and apply policy from platform
4. Model Configuration
5. Configure Schema
6. Finish