Embeddings, similarity, and outliers

Investigate distribution, nearest neighbors, duplicates, and outliers.

Embedding tools help experts find the data worth inspecting. They do not replace ontology decisions, review, or source-aware judgment.

Before you make the change

• Choose a source scope and compatible embedding space.

• Know whether the question is coverage, similarity, duplication, or anomaly discovery.

• Keep provenance and business metadata visible during interpretation.

Find Similar result cohort

Nearest-neighbor results become useful only after a human checks source context, false positives, and the intended action on the cohort.

Understand the product behavior

The embedding view projects embeddable assets into a visual space and supports drag or crop-style region selection. Users can select a cluster and review its associated thumbnails before adding, excluding, or organizing the selected assets.

Useful curation patterns include:

• locating dense duplicate or near-duplicate regions;

• sampling from visually distinct clusters;

• finding outliers and rare conditions;

• comparing source domains;

• building a more diverse pilot dataset;

• selecting negative examples near a target class.

The two-dimensional plot is a navigation aid, not a guarantee that every nearby point is semantically identical. Operators should inspect the actual assets before turning a region into a dataset.

Custom embedding spaces and vector search

The SDK can create named embedding spaces with a specified dimensionality and optional model name, upload vectors for an asset or a video frame, bulk-upsert vectors, search by a query vector, and optionally scope the search to a project or level.

This allows teams to use Unitlab’s curation interface while retaining control over the embedding model and vector space used for similarity operations.

Search, duplicates, outliers, and UMAP

Unitlab includes visual similarity search, natural-language CLIP-style search, duplicate detection, outlier detection, quality-inconsistency checks, and UMAP visualization. These embedding and visual exploration experiences are available for image and video data.

Turn vector evidence into a controlled cohort

1

1. Inspect global distribution

Open Embedding view before zooming into a cluster so the local region has context.

2

2. Choose a seed or region

Select a representative item, freehand region, or query vector.

3

3. Review candidates

Compare neighbors, duplicates, or outliers against metadata and the native asset viewer.

4

4. Separate signal from decision

Remove false positives and document the criterion used by the human reviewer.

5

5. Version the accepted cohort

Tag, group, or publish the verified membership as required by the operating flow.

Decisions that affect production

Decision	Production guidance
Embedding model	Record the selected space because results can change across models or versions.
Distance threshold	Use as a retrieval control, not as an unreviewed acceptance rule.
Duplicates	Decide whether to remove, group, down-weight, or retain them for a documented reason.
Outliers	Review for rare value, corruption, distribution shift, or ingestion failure before acting.

Continue the operating flow

• Create a balanced dataset version.

• Use the cohort for annotation, review sampling, or model evaluation.

• Retain the embedding and source version in the decision record.

Product context

Unitlab helps AI teams curate, annotate, manage, version, and prepare multimodal training data at enterprise scale.

See Unitlab’s multimodal data annotation platform for the commercial overview of curation and annotation in one workflow.

---

Explore related Unitlab capabilities: multimodal data curation