Question 1

What is a three-tier QA cascade?

Accepted Answer

A three-tier QA cascade is Corpshore's quality process where every unit of annotated data passes three independent checks before it ships: annotator plus peer review, an expert QA-lead audit, and programmatic consensus and gold-set benchmarking. Each tier catches what the previous one missed, which is how the process reaches 97%+ accuracy at scale.

Question 2

How does Corpshore measure accuracy?

Accepted Answer

Accuracy is measured against gold sets, reference data whose correct labels are known, and against consensus scoring across annotators. Inter-annotator agreement is tracked over time and fed back into annotator guidance, so quality is a measured number rather than a claim.

Question 3

Does the cascade slow delivery down?

Accepted Answer

No. The tiers run as an integrated pipeline rather than sequential gates that stall the work. Programmatic checks and statistical sampling keep throughput high while catching errors early, so quality and cadence hold together.

Question 4

How do you reach 97% accuracy?

Accepted Answer

The figure comes from layering independent checks so no single reviewer's error passes through unnoticed. Annotator and peer review catch most mistakes, the QA-lead audit catches systematic ones, and programmatic gold-set and consensus checks catch what humans overlook, gating the batch before it ships.

Question 5

What is a gold set and how do you use it?

Accepted Answer

A gold set is a sample of data with known-correct labels agreed by experts. Corpshore seeds gold items into live work to measure each annotator's accuracy in real time, and uses gold-set performance to gate batches and to target retraining where agreement drops.

Question 6

What happens when the QA cascade finds an error?

Accepted Answer

Flagged units are corrected and, where the error points to a pattern, the annotator guidance or taxonomy is clarified so the same mistake does not recur. Systematic issues trigger a re-review of the affected batch rather than a one-off fix, keeping the whole dataset consistent.

Question 7

How do you keep quality consistent across languages and hubs?

Accepted Answer

Every hub runs the same three-tier cascade against the same gold-set and agreement standards, and native, in-region annotators handle their own languages so quality does not degrade on non-English data. Central QA leads audit across hubs to keep one bar for the whole program.

Question 8

Can you meet quality requirements for regulated or high-stakes data?

Accepted Answer

Yes. For regulated work, credentialed reviewers annotate under a documented protocol with full provenance capture and tiered sign-off, producing an auditable chain of custody. That is how Corpshore has delivered datasets that stood up to regulatory scrutiny in healthcare imaging.

How we lock in 97%+ accuracy

Annotator + peer review

Expert QA lead

Programmatic + consensus

Inject an error, watch the cascade catch it

Tier 1 · Annotator + peer

Tier 2 · Expert QA lead

Tier 3 · Programmatic + consensus

Quality, answered

Ready to scope a pilot?