# Determinism

> The same request bytes to the same GPU model version return the same answers, byte for byte. Scope, limits, method and checks.

On a GPU model version, the same request always returns the same answers, byte for byte. You can snapshot-test against Dex, replay a request to audit a past decision, and retry without getting a different result.

## The guarantee

On the GPU path, identical request bytes sent to the same exact model version return byte-identical `answers`, `model`, `calibration` and `usage`. This holds:

- whether you name the exact version (`dex-1.0.0`) or the alias (`dex-1`), as long as the alias resolves to that version;
- across time and across load;
- whatever other requests are on the GPU at the same moment. Other requests never change your result.

Send the same bytes and you get the same bytes back. Keep your request body fixed in your tests, including the order of keys: reordering options, or the objects of a JSON state, is a different request.

## What v1 does not promise

- **Question isolation.** The questions of one request are computed together after the shared state. Adding, removing or reordering another question in the same request can move an answer slightly, because the GPU rounds differently when the packed input gets longer or shorter. The same request bytes still give the same answers every time. Computing each question exactly as if it were asked alone is on the roadmap.
- **Equivalent spellings.** The guarantee is for identical request bytes. Bodies that differ only in whitespace or in Unicode normalization render the same text for the model, so their probabilities match in practice. Labels and levels are echoed exactly as you sent them, so a label in a different Unicode form comes back in that form.

If you need an answer that cannot depend on the other questions, send that question in a request of its own.

## What differs between calls

- `id` and `created` are new on every call. An [idempotent replay](/docs/reference/idempotency/) returns the original ones.
- Response headers such as `x-request-id` and the rate limit headers change.

## Scope

The guarantee covers GPU versions only.

- **Fallback versions are excluded.** The hosted fallback is not bit-deterministic. Its responses say so: `served_by` is `fallback`, `model` is a `dex-fallback-*` version, and `GET /v1/models` lists it with `deterministic: false`. See [Fallback and served_by](/docs/concepts/fallback/).
- **Pinning an exact version keeps you on the GPU.** A request that names an exact GPU version is never sent to the fallback. If no GPU can serve it, it gets `503`.
- **`fallback: "never"` does the same for the alias.** The request stays on the GPU path and gets `503` when no GPU can serve it.
- **The alias moves.** When a new version ships, `dex-1` resolves to it, and answers can change. The [changelog](/docs/changelog/) records every move. To hold answers fixed over time, pin an exact version. See [Models and versions](/docs/concepts/models/).

## How it is achieved

GPU inference is usually not bit-reproducible: the order of floating-point additions can change with batch size, and that changes the last bits of the result. Dex removes each source of variation that other requests could bring in.

1. **One request per GPU pass, or batch-invariant kernels.** The engine serves one request per pass. When it batches requests, it uses kernels whose sums run in the same order no matter how many requests share the pass. Kernel settings are fixed when the engine is built, not tuned at run time.
2. **Shared state, packed questions.** The state is read once. The questions follow it in one packed input, and a mask keeps each question from reading the others. The packed length still affects rounding, which is why v1 does not promise question isolation.
3. **Fixed padding and order.** Inputs are padded to fixed sizes and laid out in a fixed order: prefix, state, then questions. The engine only ever runs a small set of shapes, all compiled in advance.
4. **No sampling.** Answers come from the model's scores in one pass over the input. There is no random number generator anywhere on the path.
5. **Pinned profile and golden gate.** An exact version fixes the weights, tokenizer, prompt template, the inference software and hardware configuration, and the calibration. When the GPU worker starts, it runs a suite of 500 golden requests for each version and compares a hash of the raw scores with the version's stored golden hash. On a mismatch, for example after an unplanned driver update, it does not serve that version and raises an alert.
6. **One post-processing implementation.** Temperature scaling, softmax, rounding and confidence run in one place, in the gateway, in 64-bit floating point. Fixed test vectors (known scores in, known bytes out) must pass before the gateway can deploy.

## How it is verified

- **A determinism suite in CI, on the real GPU.** Every golden request runs alone, 20 times in a row, and mixed with other requests under load. Every response must be byte-identical each time. Shuffled question orders and single-question subsets are measured and reported for the isolation roadmap item.
- **A production canary every 5 minutes.** A fixed synthetic request, with no customer data, runs against every live version and is compared with its stored bytes. A mismatch pages the on-call engineer and takes the affected inference node out of service.

Results of both are published at launch.

## Using it

- **Snapshot tests.** Pin an exact version, send fixed request bodies with a test key, and store the responses. Compare `answers` and `usage` on later runs. Test keys always use the GPU path, and they are free up to 250,000 tokens a day.
- **Audits.** To explain a past decision, send the same request bytes to the same exact version. Store the request, the `model` and the `calibration` from the response.
- **Safe retries.** With an [idempotency key](/docs/reference/idempotency/), a retried request is charged at most once and returns the original response.
- **Upgrades.** When a new version ships, run your snapshots against it before you move your pin. Superseded versions stay available for at least 90 days.