> For the complete documentation index, see [llms.txt](https://docs.unitlab.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.unitlab.ai/documentation/qa/qa-workflows.md).

# QA Workflows

Connect annotation, consensus, benchmark gates, review, and rework with explicit owners and outcome routes.

A QA workflow connects quality controls to the work they govern. Choose the checks the task requires, assign accountable owners, and define what happens when work passes, fails, lacks comparison evidence, or needs a human decision.

Use [QA Overview](https://docs.unitlab.ai/documentation/qa/qa-overview) to select controls. Use [Stages and routes](https://docs.unitlab.ai/documentation/workflows/stages-and-routes) for the full stage catalog and editor behavior.

## See the workflow in action

The demo shows a workflow containing Annotate, Consensus, Quality Gate, Review, and Complete, with correction routes and gate configuration.

{% embed url="<https://homepage-files.s3.us-east-2.amazonaws.com/hero-videos/hero/annotation-quality-assurance-1-20260918.mp4>" %}

[Open the demo in a new tab](https://homepage-files.s3.us-east-2.amazonaws.com/hero-videos/hero/annotation-quality-assurance-1-20260918.mp4).

Routing requirement: connect Consensus Fail to a Review stage. Configure Quality Gate Pass, Fail, and Not evaluated separately, as shown in the example below.

## Start with the smallest sufficient process

| Pattern                                                               | Use when                                                                                          | Essential exception route                                              |
| --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
| **Project → Annotate → Review → Complete**                            | Every item needs human acceptance.                                                                | Reject returns to Annotate.                                            |
| **Project → Consensus → Review → Complete**                           | Independent judgments and expert resolution are required.                                         | Disagreement reaches an eligible reviewer.                             |
| **Project → Annotate → Quality Gate → Review → Complete**             | Approved references should inform the process.                                                    | Fail and Not evaluated each have an explicit destination.              |
| **Project → Annotate → Consensus → Quality Gate → Review → Complete** | The process requires prepared labels, independent comparison, reference checks, and final review. | Disputed, failed, and unavailable results remain owned and observable. |

These are example designs. Consensus itself collects human annotations; an earlier Annotate stage is optional. Existing annotations can seed the independent assignments, so decide whether seeded work fits the independence required by your quality policy.

## Example: layered checks with final review

```mermaid
flowchart TD
  P[Project] --> A[Annotate]
  A --> C[Consensus]
  C -->|Agreement| G[Quality Gate]
  C -->|Needs review| R[Review]
  G -->|Pass| R
  G -->|Fail| A
  G -->|Not evaluated| R
  R -->|Approve| F[Complete]
  R -->|Reject| A
```

In this example, reviewers accept the final result whether it reached them through agreement, a passed reference check, or unavailable benchmark evidence. Failed benchmark work and rejected reviews return to Annotate. If your policy requires every reviewed Consensus result to pass a gate before completion, configure that additional route explicitly and pilot it; the example above does not imply it.

## Build the workflow

1. **Write the policy first.** Define label quality, required checks, who can accept work, and how unbenchmarked items should be treated.
2. **Prepare inputs.** Confirm source readiness, ontology, instructions, and representative approved answer keys where needed.
3. **Add stages.** Create only stages that represent a distinct responsibility or automatic check.
4. **Assign people.** Set annotator and reviewer eligibility and verify the available self-assignment or manager controls.
5. **Set check parameters.** Configure Consensus votes and similarity, and the Quality Gate threshold. Calibrate them with actual examples.
6. **Connect every outcome.** Include the disagreement route, gate Pass/Fail/Not evaluated, and review approval/rejection paths.
7. **Save and pilot.** Inspect the impact of **Save & Apply**, then validate the operating scenarios below.

## Validate the process before scaling

| Pilot case                      | Expected evidence                                                         |
| ------------------------------- | ------------------------------------------------------------------------- |
| Complete, acceptable annotation | Reaches the required next stage with the right owner.                     |
| Insufficient Consensus votes    | Waits for the remaining votes rather than passing early.                  |
| Consensus agreement             | Publishes the representative result and follows the agreement route.      |
| Consensus disagreement          | Opens the intended review path with the submitted evidence available.     |
| Benchmark score above threshold | Uses the configured Pass route.                                           |
| Benchmark score below threshold | Uses the configured Fail route.                                           |
| No usable approved key          | Uses Not evaluated rather than reporting a successful benchmark check.    |
| Rejected review                 | Returns to correction with actionable feedback and can be reviewed again. |
| Grouped work                    | Treats the group as one task while preserving required-panel evidence.    |

Inspect both the queue and the Workbench after transitions. A correct-looking graph is insufficient if work cannot be claimed, a reviewer lacks access, or a correction loop does not return to the required check.

## Monitor the process

Use **QA → Benchmarks** to track key readiness and evaluations, **QA → Consensus** for agreement and disputed results, and **QA → Quality checks** for validation problems and open issues. Use queues and item history to identify the current owner and the sequence of attempts.

Automatic Quality Gate evaluation is not a human assignment queue. If an item appears stalled, inspect the current stage and any processing or comparison issue before assigning it to a person.

## Change a running process deliberately

Review in-flight items before removing stages, changing routes, altering Consensus requirements, or replacing a project workflow. Preserve active vote and review work where the product allows it; some runtime settings are locked while Consensus items are active. Changing eligibility is different from explicitly reassigning an existing task.

Follow [Change a live workflow](https://docs.unitlab.ai/documentation/workflows/change-a-live-workflow), record the intended handling of active work, and repeat the affected pilot scenarios after applying the change.

## Release the accepted result

Confirm that the intended cohort has completed all required checks and that unresolved findings have an explicit disposition. A dataset membership version and an annotation release serve different purposes: the former defines selected source data; the latter freezes reviewed project output. Verify representative exported items, ontology values, relations, and source identity before delivery.

## Next steps

* [Consensus](https://docs.unitlab.ai/documentation/qa/consensus)
* [Quality Gate](https://docs.unitlab.ai/documentation/qa/quality-gate)
* [Review Stages](https://docs.unitlab.ai/documentation/qa/review-stages)
* [Create a release](https://docs.unitlab.ai/documentation/releases/create-a-release)