Every endpoint, scored in the open.
242 of 242 field-level checks passing across 37 endpoints — run against the live production API on 2026-07-08, not a lab build. The same suite gates every prompt and routing change before it ships.
Documents
Extract & structure
Commerce & ops
Communication
Content & moderation
Web & research
Media & agents
Kynth vs the field, on public datasets.
An MIT-licensed, CI-run benchmark against AWS Textract and Google Document AI on pre-registered public document sets — raw API responses committed, every number reproducible with your own keys or with none at all. Including the cells where Kynth loses.
| Suite | Metric | Kynth Apollo | AWS Textract | Google DocAI |
|---|---|---|---|---|
| Invoices(fatura, n=100) | field accuracy | 99.4% ◆ | 92.2% | 93.1% |
| Receipts(sroie, n=100) | field accuracy | 88.8% ◆ | 68.8% | 52.8% |
| Receipt line items(cord, n=100) | line-item F1 | 41.9% | 77.1% ◆ | 36.6% |
| Receipt totals(cord, n=100) | totals accuracy | 67.1% | 95.5% ◆ | 82.2% |
| Tables(fintabnet, n=100) | TEDS | 0.791 | 0.836 ◆ | 0.365 |
Full pre-registered subsets, verbatim raw responses, and methodology: github.com/kyisaiah47/doc-extract-bench (v0.1.0, run 2026-07-10). Textract wins CORD line-items and tables — published unedited; Kynth’s 14 table failures are counted against it, not dropped. Veryfi and LlamaParse adapters exist in the harness and run the moment credentials are supplied. Bank statements are pending an upstream fix to the third-party evaluator.
How this is measured
Each endpoint has a golden suite of realistic inputs with field-level expectations — exact values, dates, totals, enums, and redaction checks, never vibes about prose. The runner calls the production API with a real key and scores every expectation individually. Media-only endpoints (transcribe, describe) are verified with generated audio and images through the same production path. The suite re-runs on every prompt, model-routing, or endpoint change; an endpoint that drops below threshold doesn’t ship.