Method

Our method.How we know the data is right.

Quality is measured, not promised. Every expert is calibrated before live work, checked against gold answers during it, and every batch passes QA review before it ships.

Quality · clinical-evalsSL
Items2,400
Passed QA96.4%
Gold agreement0.91
Redone86
Gold agreement by batch
Item #A-20931

Failed gold check: missed a renal dose adjustment.

RedoneNot billed
Illustrative interface

Calibrate before live work

Before an expert touches a live task, they complete a calibration set: tasks with known answers, written for the project. Experts who do not meet the bar on calibration do not start.

Check against gold during the work

Gold items with known answers are mixed into live work. Agreement with gold is tracked per expert, so drift is caught early and the expert is re-calibrated or removed from the project.

Review every batch

Each batch goes through QA review before delivery. Items that fail are redone or rejected, and you are not billed for them. Across our work, 96%+ of items pass QA review.

Report what we found

Every pilot and batch comes with a QA report: what was checked, what failed and why, and how agreement looked across experts. Where experts disagree on a hard item, we say so rather than hide it.

What this cannot do

Calibration and gold checks raise the floor and catch drift. They do not make experts infallible. Expert data should still be evaluated like any other data, and we plan our QA with that in mind.

Questions, answered.

What does the 96%+ acceptance rate measure?

The share of delivered items that pass QA review on the first pass, across our work.

Can we set our own QA bar?

Yes. The QA bar and agreement threshold are part of the written spec for each engagement.

Every safe model
has an expert behind it.