Quality
Calibration reports
Measured calibration and agreement per model version.
Every exact model version gets a calibration report, one per calibration version. Each response names its calibration version in the calibration field, so every answer traces back to the numbers here. See Calibration for how they are made, the data behind them and its known limits.
Current measurements
Model dex-1.0.1, calibration cal-20260926-1. Measured 2026-09-26 on Dex's production path in the EU. Evaluation set: Dex decision eval v1, test split: 300 items, 921 questions, English and Dutch; reference = average of gpt-5.3-chat and claude-opus-5 (they agree on 96.3%); temperatures fitted on the 100-item dev split.
| Measurement | Value |
|---|---|
| Agreement with reference labels, English and Dutch (921 questions) | 82.3% |
| Agreement, English (465 questions) | 84.9% |
| Agreement, Dutch (456 questions) | 79.6% |
| Agreement, pick (400 questions) | 83.0% |
| Agreement, check (353 questions) | 90.4% |
| Agreement, rate (beta) (168 questions) | 63.7% |
| Calibration error (ECE), all questions (921 questions) | 0.044 |
| Calibration error (ECE), pick (400 questions) | 0.048 |
| Calibration error (ECE), check (353 questions) | 0.052 |
| Calibration error (ECE), rate (beta) (168 questions) | 0.138 |
| Latency p50, GPU path, end to end | Published at launch |
| Latency p95, GPU path, end to end | Published at launch |
| Determinism canary mismatches | Published at launch |
For rate, agreement counts the most probable level. Counted on rating rounded to the nearest level, it is 58.3%. The rating is off by 0.591 levels on average.
Percentages are rounded to one decimal and ECE to three, and a value exactly halfway is rounded down.
rate questions are in beta: their agreement and calibration error miss our bars. See Quality bars.
Versions
dex-1.0.0, calibrationcal-20260925-1: measured 2026-09-25 on Dex's production path in the EU.dex-1.0.1, calibrationcal-20260926-1(current): measured 2026-09-26 on Dex's production path in the EU.
For dex-1.0.1 this page shows agreement and calibration error with the number of questions behind them, and a reliability curve per question type. ECE and curves per language, bootstrap bands, Brier scores and the fitted temperatures are not published yet. Fallback versions have no report yet.
Reliability curves
Reliability diagram, dex-1.0.1. pick: ECE 0.048, agreement 83.0%; check: ECE 0.052, agreement 90.4%; rate (beta): ECE 0.138, agreement 63.7%.
pick- ECE
- 0.048
- agreement
- 83.0%
- questions
- 400
check- ECE
- 0.052
- agreement
- 90.4%
- questions
- 353
ratebeta- ECE
- 0.138
- agreement
- 63.7%
- questions
- 168