Video Annotation

Create frame-accurate object tracks, temporal events, dynamic properties, and synchronized video annotations at scale.

Video annotation adds time, identity, and change to visual labeling. Unitlab combines frame-level geometry, full or directional auto-tracking, interpolation, synchronized audio, dynamic properties, and a complete timeline so teams can review what happened, when it happened, and which object remained the same.

Use this guide when: you are building object-tracking, action, event, behavior, scene, or multimodal video datasets.

See video annotation in action

The demo shows frame-aware visual annotation and AI assistance. Use the timeline—not playback alone—to inspect track continuity and temporal labels.

Open the demo in a new tab.

Before you begin

1. Create or select a project whose data and ontology match this modality.

2. Confirm the project instructions define the unit of annotation, boundary or timing policy, required properties, and review route.

3. Open the project and enter the assigned item from the project data view or queue. The Workbench loads the modality-native editor inside the shared Unitlab shell.

See Annotation Workbench for navigation, saving, item state, comments, issues, and workflow actions.

Understand the video work surface

The video Workbench combines the visual canvas with playback controls, frame navigation, an annotation timeline, ontology controls, and item workflow actions. Geometry can be created on an exact frame, propagated with tracking or interpolation, and corrected at keyframes. Audio can remain synchronized to the video when sound is part of the decision.

Three cyclist frames aligned to an exact keyframe timeline

Frame-accurate labeling keeps the visual state and the timeline state aligned.

Supported annotation model

Annotation type	Use it for
Tracked box, polygon, mask, or cuboid	Object localization that persists across frames.
Keypoint or skeleton track	Pose or landmark motion over time.
Line or polyline	Temporal paths and elongated structures.
Temporal classification or event	An action, scene, state, or interval on the timeline.
Dynamic class property	A property of one tracked object that changes over time.
Dynamic Item Property	A whole-video state or event that changes over time.
Relation	A governed connection between tracked objects.
Static Item Property	A fact that applies to the complete sequence.

Tracking predicts object state across frames; interpolation fills geometry between explicit keyframes. They solve different problems and both require timeline review.

Full, forward, backward, and multi-object tracking

Create a reliable seed annotation, then run the tracking direction that matches the visible interval. Full tracking covers both directions around the seed; forward or backward tracking limits the prediction range. Auto-Track All can propagate multiple selected objects together. Correct identity switches, drift, missed reappearances, and shape errors at the first frame where they occur.

A van and cyclist selected and tracked together across video frames

Multi-object tracking accelerates propagation while the annotator remains responsible for identity and geometry.

Dynamic properties and Item Properties

Use dynamic class properties when the state belongs to one tracked object—for example vehicle motion or visibility. Use dynamic Item Properties when the state describes the complete video—for example scene condition or global event. Create explicit temporal segments and review their start and end frames. A property value outside its intended time range is a label error even when the geometry is correct.

Timeline review for long sequences

Use the overview timeline to find tracks, keyframes, temporal segments, gaps, and dense regions. Zoom into a local interval for exact correction, then return to the full sequence to confirm continuity. Long videos should be reviewed at transition points, object entrances and exits, occlusion, shot changes, and every property boundary.

Video moments connected to tracks, states, keyframes, and a shared playhead

The timeline is the authoritative map of track continuity, temporal state, and review coverage.

Annotate one production item

1

1. Find the first reliable frame

Navigate to a frame where the target is visible and unambiguous. Confirm the ontology class and the intended identity rule.

2

2. Create the seed geometry

Draw the required box, polygon, mask, cuboid, keypoints, or skeleton and complete any properties that apply at the seed.

3

3. Choose propagation

Use full, forward, or backward auto-tracking for model-based propagation, or add explicit keyframes and interpolation when controlled geometric transition is the correct method.

4

4. Correct the first failure

Scrub the timeline and stop at the first drift, identity switch, occlusion error, or missed reappearance. Correct there before continuing so later predictions do not inherit the mistake.

5

5. Add temporal meaning

Create dynamic class properties, dynamic Item Properties, events, or classifications with exact start and end frames. Add object relations where the ontology requires them.

6

6. Review and route

Inspect the full timeline, key transitions, synchronized audio, required values, and final object identity; then save and use the configured stage action.

Quality review

Review focus	What to check
Identity	One track must represent one real object; split or repair identity switches.
Frame boundaries	Check the exact first and last valid frames for every track and event.
Geometry drift	Review changes in scale, pose, occlusion, motion blur, and camera movement.
Temporal values	Confirm dynamic properties and Item Properties change only on the intended frames.
Audio context	When audio informs the label, review playback and waveform context together.

A saved annotation is not automatically a production-ready annotation. Required values, boundary or timing policy, cross-item consistency, and the configured review stage still apply.

Move from labels to governed data

Video outputs should preserve track identity, keyframes, temporal ranges, dynamic properties, relations, and the source frame rate or timestamp basis. Route model-assisted results through the same review stage as manual work and version material changes to tracking or ontology policy.

Integrated Unitlab workflow connecting model assistance, annotation, review, and quality assurance

Use workflows to keep model output, human correction, review, and approval in one traceable operating path.

Next steps

• Use AI-assisted Annotation to calibrate interactive and batch assistance.

• Use Multimodal overview when related files or views must stay in one task.

• Curate difficult cases and review cohorts in Data curation.

• Read the current video annotation product overview for the feature overview and current media.