Idempotency

When a request times out or the connection drops, you cannot tell whether it ran. Send an idempotency-key header, and you can retry with the same key without being charged twice. A replay of a completed request is never charged.

How to use it#

Send the header on POST /v1/decide. The console's top-up route accepts it too.

curl https://api.thinqit.ai/v1/decide \
  -H "authorization: Bearer $DEX_API_KEY" \
  -H "content-type: application/json" \
  -H "idempotency-key: 7f3c1e9a-2b4d-4c8e-9a61-0d5f2e8b7c43" \
  -d @request.json

• A key is 1 to 255 printable ASCII characters, without spaces. A malformed key gives 400 invalid_header.
• A UUID is a good choice.
• Use one key per logical request, and send the same key on every retry of that request.
• Use a new key for a new request, even when its body is the same as an earlier one.

What happens to the key#

• A key is scoped to the API key that sent it (to the account, for console routes). Two of your API keys can use the same idempotency key without clashing.
• Dex keeps each idempotency key for 24 hours.
• It stores a small record: your key id, a hash of the idempotency key, a fingerprint of the request (a SHA-256 hash of its canonical form), the status and timestamps. A completed request adds its original id and created, the model version, served_by, the token counts and the charge fields. It stores no request content and no answers.

Replays#

When a request with the same key and the same body has already completed, you get the original response, and nothing is charged again:

• The response carries idempotent-replayed: true.
• id, created and usage are the original ones, including the original charge fields. They show what the first call cost; nothing is debited now.
• The x-request-id header names the replay itself, so support can find both calls.
• On a GPU version, the answers are computed again on the original exact version and are the same, byte for byte, as the first time. See Determinism. A replay never goes to the fallback: if no GPU can serve that version, you get 503 no_capacity with retry-after, and nothing is charged.
• One exception: an original served by the fallback. Its answers cannot be reproduced and are not stored. The replay runs the request again on the normal path, without charging. Its answers, model and served_by can differ from the original's. id, created and usage are still the original's.

In short: idempotency makes sure you are charged at most once, and a replay looks exactly like the call it retries.

Conflicts#

After a failure#

• 5xx. A request that ended in a 5xx is recorded as failed, and its charge was refunded. A retry with the same key runs again normally, and it is charged once if it succeeds.
• 429 and 402. A request stopped by a rate limit or by your balance keeps no record. A retry with the same key, after retry-after or after a top-up, runs as a new request.
• Other 4xx. A 400, 401, 403, 404, 413 or 422 is rejected before the idempotency step and never creates a record. Fix the request, and use a new key for the fixed request, because its body is different.

A 4xx is never charged.

Idempotency records live in a fast store without persistence in v1. A failover of that store forgets them. While it is unreachable, a decide request that carries an idempotency key gets 503 billing_unavailable, so a retry can never be charged twice.

What counts as the same body#

Dex compares requests by their canonical form: the parsed JSON body with object keys in their original order, no insignificant whitespace, strings in Unicode NFC form and numbers in their shortest form.

• Reformatting the JSON, such as adding spaces or line breaks, does not change the fingerprint.
• Changing the order of keys, including the options of a pick, does change it.

SDKs#

Until the SDKs are on PyPI and npm, install them from Downloads.

The SDKs send an idempotency key on every decide call without being asked. They use your key if you pass one, or a new UUID v4 otherwise, and reuse it on every retry of that call. Read-only calls send none.

Pass your own key when a retry can happen in a different process, for example after a worker restart, so the retry still matches:

decision = client.decide(state=state, questions=questions, idempotency_key=job.id)

const decision = await client.decide({ state, questions }, { idempotencyKey: job.id });