> For the complete documentation index, see [llms.txt](https://docs.unitlab.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.unitlab.ai/documentation/auto-labeling/bring-your-own-models.md).

# Bring your own Models

Register, validate, map, secure, and operate private HTTP inference models inside Unitlab workflows.

Bring Your Own Model (BYOM) connects a private HTTP inference endpoint to Unitlab so proprietary, fine-tuned, or domain-specific models can participate in annotation and workflow automation.

![Current My AI Models catalog and Integrate External Model entry point](https://292810646-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FGjVLUz4wthGkGlRKM6rM%2Fuploads%2FTKWuIgmzJDu2lWyTXnhJ%2Fal-my-ai-models.webp?alt=media)

![Bring your model into the Unitlab annotation workflow](https://292810646-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FGjVLUz4wthGkGlRKM6rM%2Fuploads%2FOkpJdk0WWWECU4SjIe7e%2Fbring-your-model.svg?alt=media)

### Operating model

Unitlab stores the integration configuration and invokes the approved endpoint. Your team owns the model runtime, capacity, availability, version, authentication, and change control. Integrated models are private to the workspace and appear under **My AI Models**.

### Supported contracts

| Dimension          | Current support                                            |
| ------------------ | ---------------------------------------------------------- |
| Inputs             | Image, video, audio, text, medical, geospatial             |
| Visual outputs     | Bounding box, polygon, mask, skeleton, line, point, cuboid |
| Geospatial outputs | Point, line, polygon, bounding box, mask (as GeoJSON)      |
| Audio outputs      | Event and optional speech-recognition transcript           |
| Text outputs       | Entity                                                     |
| Lifecycle states   | Running, Stopped, Integration unfinished                   |
| Workflow use       | Model stage and supported assisted/batch operations        |

### Before you start

* Deploy a reachable HTTPS endpoint that accepts POST requests.
* Identify the model owner, version, and on-call owner.
* Prepare authentication headers without exposing secrets in documentation.
* Define the exact input and output schema.
* Prepare representative validation data, including failure cases.
* Create or approve the destination ontology classes and integer mappings.
* Confirm the endpoint can handle the intended batch concurrency.

{% hint style="warning" %}
Do not paste live API keys, bearer tokens, or private endpoints into screenshots, tickets, or public documentation. Use an approved secret-management and rotation process.
{% endhint %}

### Integration wizard

{% stepper %}
{% step %}

#### Registration

Open **My AI Models** and choose **Integrate External Model**. Select the generic input type and output data type, then enter the model name, description, endpoint, headers, and parameters.
{% endstep %}

{% step %}

#### Validation

Upload a representative sample. Unitlab calls the endpoint and displays the validation status and raw response. Continue only when the endpoint is reachable and the response matches the selected schema.
{% endstep %}

{% step %}

#### Integration

Add organizational tags and define output classes. Map each integer class value to the intended class name, color, and annotation geometry.
{% endstep %}

{% step %}

#### Confirmation

Review the complete contract: model identity, input type, endpoint, validation status, tags, classes, and mappings. Confirm to create the private model.
{% endstep %}
{% endstepper %}

### Representative image request

The exact payload depends on the configured data type. A typical image request contains a signed source URL and optional crop context:

{% code collapsedlinecount="10" %}

```json
{
  "src": "https://signed-source-url.example/image.jpg",
  "coordinates": [[120, 80], [920, 680]],
  "rotation": 0
}
```

{% endcode %}

| Field           | Meaning                                                        |
| --------------- | -------------------------------------------------------------- |
| **src**         | Time-limited source URL the endpoint downloads                 |
| **coordinates** | Optional crop bounds; omitted or null for full-image inference |
| **rotation**    | Source orientation in degrees                                  |

A representative bounding-box response maps each result to an integer class:

{% code collapsedlinecount="10" %}

```json
{
  "bboxes": [[
    {
      "point": [[100, 50], [300, 50], [300, 240], [100, 240]],
      "class": 0
    }
  ]],
  "classes": ["person"]
}
```

{% endcode %}

Treat these examples as a contract starting point. The Validation step is authoritative for the selected input and output type.

### Geospatial models

Connect a model that labels satellite, aerial, or drone rasters. In **Integrate External Model**, choose **Geospatial** as the input type. Unitlab sends your endpoint the whole raster, and your model answers with GeoJSON.

#### What Unitlab sends

A `POST` with a JSON body: your saved parameters, plus these keys.

| Key         | Meaning                                                                                                                                                                |
| ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **src**     | Signed link to the raster. In a workflow, the whole original raster, valid for 7 hours. In the model test, the sample file you picked; download it during the request. |
| **output**  | Asks for GeoJSON (RFC 7946) and names the raster's coordinate reference.                                                                                               |
| **unitlab** | Workflow only: `run_id`, `result_upload_url`, and `result_upload_token`, used to answer later.                                                                         |

#### What your model returns

Answer with HTTP 200 and a GeoJSON `FeatureCollection`:

{% code collapsedlinecount="10" %}

```json
{
  "type": "FeatureCollection",
  "classes": ["building", "tree"],
  "features": [
    {
      "type": "Feature",
      "geometry": {
        "type": "Polygon",
        "coordinates": [[[69.2401, 41.3111], [69.2405, 41.3111], [69.2405, 41.3108], [69.2401, 41.3108], [69.2401, 41.3111]]]
      },
      "properties": {"class_name": "building", "confidence": 0.93}
    }
  ]
}
```

{% endcode %}

* **Classes.** List them in `classes` when you answer the model test. A passing test saves that list as the model's classes. In workflow runs, each feature names one of them with `class_name`, or with `class` as its position in the list, starting at 0.
* **Coordinates.** Use longitude and latitude (WGS84). The raster's own map units and pixel positions are also read. A shape that crosses the raster's edge is clipped to it; a shape entirely outside the raster is dropped.
* **Geometry.** `Point` becomes a point, `LineString` a line, and `Polygon` a polygon, or a bounding box or mask when `properties.markup_type` says so. Each part of a `MultiPolygon` becomes its own object.

See [GeoJSON (geospatial models)](https://docs.unitlab.ai/api-sdk-cli/apis/annotation-json-reference#geojson-for-geospatial-models) for every field.

#### Models that take longer

Answer the workflow request at once, then send the result when it is ready:

```json
{"status": "accepted", "job_id": "your-job-123"}
```

```bash
curl -X POST "$RESULT_UPLOAD_URL" \
  -H "Content-Type: application/json" \
  -H "X-Unitlab-Callback-Token: $RESULT_UPLOAD_TOKEN" \
  -d '{"run_id": "<unitlab.run_id>", "status": "complete", "result": {"type": "FeatureCollection", "features": []}}'
```

* To report a failure, send `"status": "failed"` with an `error` message.
* While Unitlab waits, the item shows **Processing** in the Model stage. After 1 hour without a result, the run shows as timed out and can be retried. A result sent later is still accepted.
* The model test cannot wait for a callback: an `accepted` answer passes the test when it lists your `classes`.

#### How results are applied

* Existing annotations stay, and your objects are added to them. A near-identical copy of an existing object of the same class is skipped.
* When the Model stage maps model classes to project classes, a class without a mapping is left out.
* On success, the item moves to the next stage. On failure, it stays in the Model stage with the error, and a manager can retry it.
* **Tile scale** and **Auto Refine** apply only to Unitlab's built-in Prompt Labelling and Find Similar models, not to your own model.

### Class mapping

For every emitted class, define:

* stable integer value;
* human-readable name;
* destination geometry;
* destination ontology class;
* confidence interpretation;
* behavior for unknown or unmapped classes.

Never silently coerce an unsupported class into another ontology label. Reject or quarantine unmapped output.

### Production readiness

| Control           | Acceptance evidence                                            |
| ----------------- | -------------------------------------------------------------- |
| Endpoint security | HTTPS, approved authentication, secret rotation owner          |
| Availability      | Health checks, timeout, retry, capacity plan                   |
| Schema            | Successful and malformed-response tests                        |
| Mapping           | Every output has an intentional destination or rejection rule  |
| Calibration       | Threshold validated on target-domain data                      |
| Human control     | Annotate/Review route and correction policy                    |
| Observability     | Correlation ID, model version, latency, status, redacted error |
| Change control    | Revalidation after model, endpoint, schema, or mapping change  |

### Use the model in a workflow

1. Open the project workflow.
2. Add or select a **Model stage**.
3. Choose the integrated private model.
4. Configure generic type, queue scope, and class mappings.
5. Route success to Annotate or Review.
6. Keep failure visible with a named owner.
7. Save and apply on a controlled cohort.
8. Monitor correction and failure rates before scaling.

### Manage the integration

From **My AI Models**, operators can:

* inspect Running, Stopped, or unfinished status;
* continue an unfinished integration;
* update endpoint or configuration;
* review tags and supported output;
* stop or retire a model under change control.

Re-run validation after material changes. Record model version and mapping version in the release provenance used for training or evaluation.

### Failure handling

| Failure                   | Response                                                    |
| ------------------------- | ----------------------------------------------------------- |
| Non-200 endpoint response | Check availability, authentication, and server logs         |
| 200 with invalid schema   | Compare the raw response with the selected output contract  |
| Empty predictions         | Distinguish a valid abstention from a model/runtime failure |
| Timeout                   | Inspect remote task state before retrying                   |
| Unknown class integer     | Stop routing and correct the class mapping                  |
| Capacity saturation       | Reduce concurrency or scale the endpoint                    |

### Related guides

* [Batch Auto-Labeling](https://docs.unitlab.ai/documentation/auto-labeling/batch-auto-labeling.md)
* [Model stages](https://docs.unitlab.ai/documentation/workflows/model-stages.md)
* [Geospatial Annotation](https://docs.unitlab.ai/documentation/annotations/geospatial-annotation.md)
* [API keys and service identities](https://docs.unitlab.ai/documentation/security/api-keys-and-service-identities.md)
* [Cloud credential governance](https://docs.unitlab.ai/documentation/security/cloud-credential-governance.md)
