# thinQit Dex: full documentation > An EU-hosted decision API. Typed answers with calibrated probabilities, the same answer every time. Source: https://thinqit.ai. Generated at build time from the docs, the API contract and the price config. --- # Dex documentation > What the decision API does, the three question types, how a call looks, and where to read next. thinQit Dex is a decision API hosted in the EU. You send a state (text or JSON) plus typed questions about it, and you get typed answers with calibrated probabilities. On GPU model versions, the same request to the same version returns the same answer every time. Requests are processed in the EU, and by default no request content is stored. You pay for input tokens only, and output is free. Sign-up and API keys are open: create a free test key in the console at https://thinqit.ai/console/. Paid top-ups open soon. The SDKs, the CLI and the VS Code extension are coming soon to their registries; until then, install them from [Downloads](/docs/reference/sdks/#downloads). You can also run a mock of the API from the published contract: see [Try it today with a mock server](/docs/quickstart/#try-it-today-with-a-mock-server). ## The three question types | Type | What it asks | What you get back | | --- | --- | --- | | `pick` | Which one of these 1 to 255 labeled options fits? | `choice`, a probability per option, `confidence` | | `rate` (beta) | Where does it sit on this scale of 2 to 10 ordered levels? | `rating`, a probability per level, `confidence` | | `check` | Is this statement true? | `probability` of yes, `confidence` | `rate` is in beta: on our test set it agrees with the reference on 63.7% of questions, below our bar of 68%. See [Questions](/docs/concepts/questions/#rate). Every answer also has `abstained`. It is `true` when you set a `min_confidence` for the question and the answer's confidence falls below it. See [Questions](/docs/concepts/questions/) and [Abstention](/docs/concepts/abstention/). ## How a call looks You send one `POST https://api.thinqit.ai/v1/decide` with a state and your questions. Each question has an id you choose, here `angry`: ```json { "state": { "ticket": { "channel": "email", "subject": "Charged twice, still no answer", "body": "This is the third time I am writing. You charged my card twice for order 4471 and nobody replies. Fix it today or I cancel my subscription." } }, "questions": { "angry": { "type": "check", "instructions": "Is the writer of {{ticket.body}} angry?", "min_confidence": 0.6 } } } ``` The response names the exact model version, the path that served it, the calibration version and what you paid. Illustrative values: ```json { "id": "req_01M5D0000000000000000000AB", "object": "decision", "created": 1792497700, "model": "dex-1.0.0", "served_by": "gpu", "calibration": "cal-20261015-1", "answers": { "angry": { "type": "check", "probability": 0.9421, "confidence": 0.8842, "abstained": false } }, "usage": { "input_tokens": 70, "state_tokens": 52, "question_tokens": 18, "charge_micro_cents": 350, "unit_price_micro_cents": 5, "tier": "t1" } } ``` `{{ticket.body}}` is a [field reference](/docs/concepts/field-references/). It points the question at one field of the state without sending that field twice. ## Where to go next Getting started: - [Quickstart](/docs/quickstart/): get a key, try the mock, make a call in curl, Python or TypeScript, and act on the answer. Concepts: - [State](/docs/concepts/state/): what you send, as a string or as JSON, and how it is billed. - [Questions](/docs/concepts/questions/): `pick`, `rate` and `check`, criteria, and question ids. - [Field references](/docs/concepts/field-references/): point a question at one field with `{{path}}`. - [Confidence](/docs/concepts/confidence/): how `confidence` is computed and what it is not. - [Abstention](/docs/concepts/abstention/): let an unsure answer say so with `min_confidence`. - [Determinism](/docs/concepts/determinism/): the same answer every time, and where that guarantee applies. - [Calibration](/docs/concepts/calibration/): what the probabilities mean and how we check them. - [Models and versions](/docs/concepts/models/): the `dex-1` alias, exact versions and their lifecycle. - [Fallback and served_by](/docs/concepts/fallback/): when a hosted fallback answers, and how to keep a request on the GPU. Reference: - [Authentication and API keys](/docs/reference/authentication/): the bearer header, scopes, `GET /v1/key` and the key checksum. - [Errors](/docs/reference/errors/): the error envelope, every status code and when to retry. - [Headers](/docs/reference/headers/): request and response headers, including rate limit headers. - [Rate limits](/docs/reference/rate-limits/): limits per key and per account, and what a 429 means. - [Idempotency](/docs/reference/idempotency/): retry safely without paying twice. - [Limits](/docs/reference/limits/): every size limit and the error it returns. - [Tokens and billing](/docs/reference/billing/): what counts as a billable token and how the balance works. - [Data handling and residency](/docs/reference/data-handling/): where requests are processed and what we keep. - [SDKs and CLI](/docs/reference/sdks/): the planned Python and TypeScript SDKs and the command line tool. - [API reference for decide](/docs/api/decide/): the full request and response schema from the contract. - [Pricing](/pricing/) and the [changelog](/docs/changelog/). --- # Quickstart > Get a key, try the API against a local mock today, make a call in curl, Python or TypeScript, and act on the answer. This page takes you from nothing to a working decision about a support ticket: which team should handle it, how fast to reply, and whether the writer is angry. Sign-up and API keys are open: sign in to the console with your email address and create a free test key. Paid top-ups open soon. The SDK packages are coming soon to PyPI and npm; until then, install them from [Downloads](/docs/reference/sdks/#downloads). You can also try everything against a local mock server first. ## Get an API key Keys come from the console at https://thinqit.ai/console/. Sign in with your email address; the first sign-in creates your account. There are two kinds: - **Live keys** start with `dex_live_`. Calls are billed against your prepaid balance. - **Test keys** start with `dex_test_`. Calls are free up to 250,000 tokens a day per account. Test calls always run on the GPU path and have lower rate limits. The console shows a key's secret once, when you create it. Store it in an environment variable, where the SDKs and the examples below look for it: ```bash export DEX_API_KEY="" ``` ### Check your key Before your first decision, confirm the key works and see which account it belongs to. `GET /v1/key` works with any valid key and never returns the secret. ```bash tab="curl" curl https://api.thinqit.ai/v1/key \ -H "authorization: Bearer $DEX_API_KEY" ``` ```python tab="Python" from thinqit_dex import Client key = Client().get_key() print(key.prefix, key.mode, key.scopes) ``` ```ts tab="TypeScript" import { Client } from "@thinqit/dex"; const key = await new Client().getKey(); console.log(key.prefix, key.mode, key.scopes); ``` ```bash tab="CLI" dex whoami ``` You get the key's id, its display prefix (such as `dex_test_R4tv`), `live` or `test`, its scopes and the account id. See [Authentication and API keys](/docs/reference/authentication/). ## Write the request A request has a `state` (the material to decide about) and `questions` (what you want to know about it). Save this as `request.json`: ```json { "state": { "ticket": { "channel": "email", "subject": "Charged twice, still no answer", "body": "This is the third time I am writing. You charged my card twice for order 4471 and nobody replies. Fix it today or I cancel my subscription." } }, "questions": { "team": { "type": "pick", "instructions": "Which team should handle {{ticket.body}}?", "options": { "billing": "Payments, refunds and invoices", "technical": "Bugs, outages and sign-in problems", "shipping": "Delivery and returns", "other": null }, "min_confidence": 0.5 }, "urgency": { "type": "rate", "instructions": "How fast must we reply to {{ticket.body}}?", "levels": ["Can wait", "This week", "Within two working days", "Today"] }, "angry": { "type": "check", "instructions": "Is the writer of {{ticket.body}} angry?", "min_confidence": 0.6 } } } ``` What each part does: - `state` is JSON, so each question can point at one field with `{{ticket.body}}`. See [Field references](/docs/concepts/field-references/). - `team` is a `pick`: choose one option. `other` has no description because its label says enough. - `urgency` is a `rate`: place the ticket on four levels, listed from lowest to highest. `rate` is in beta; see [Questions](/docs/concepts/questions/#rate). - `angry` is a `check`: the probability that the statement is true. - `min_confidence` tells an unsure answer to abstain instead of guessing. See [Abstention](/docs/concepts/abstention/). - The ids `team`, `urgency` and `angry` are yours. The model never sees them. The request has no `model` field, so it uses the default alias `dex-1`. ### Autocompletion in your editor A JSON schema for request bodies is published at https://thinqit.ai/schemas/dex-request.schema.json. Point your editor at it to get completion, hover docs and errors for unknown or misspelled fields while you write a request file. In a file you run with `dex decide --file` or with the VS Code extension, add a `$schema` line at the top: ```json { "$schema": "https://thinqit.ai/schemas/dex-request.schema.json", "state": { "ticket": { "body": "..." } }, "questions": {} } ``` The CLI and the VS Code extension remove `$schema` before they send the request, and the extension applies the schema to every `*.dex.json` file by itself. When you send a body yourself, with curl or an SDK, leave `$schema` out: the API rejects fields it does not know with `422 unknown_field`. ## Try it today with a mock server Prism is an open source mock server. It reads the published API contract, checks each request against it, and answers with the contract's example responses. You need Node.js. Start the mock: ```bash npx @stoplight/prism-cli mock https://thinqit.ai/openapi.yaml ``` Prism listens on `http://127.0.0.1:4010`. In a second terminal, send the request: ```bash curl http://127.0.0.1:4010/v1/decide \ -H "authorization: Bearer dex_test_mock" \ -H "content-type: application/json" \ -d @request.json ``` What to expect from the mock: - **A valid request** gets `200` and the contract's first example: the answer to this page's ticket (`team` is `billing`), the same values as in [Read the answer](#read-the-answer) below. The mock does not run a model, so it gives these answers whatever you send. Use it to test your request shape and your response handling. - **The other examples.** Add `-H "prefer: example=dutchSupport"` for the Dutch e-bike ticket from [Abstention](/docs/concepts/abstention/#example-the-dutch-e-bike-ticket), or `-H "prefer: example=englishModeration"` for the English moderation example. - **A request that breaks the contract** gets a `4xx`. The `sl-violations` response header lists what is wrong (add `-i` to curl to see headers). The body is the contract's example error, so read the header, not the body. - **No `authorization` header** gets `401`. Prism checks that a bearer token is present, not what it is. - **Example headers.** Prism also returns example headers such as `idempotent-replayed: true`, which the real API sends only when a request is replayed. To run the Python and TypeScript code below against the mock, set `DEX_BASE_URL=http://127.0.0.1:4010` and any `DEX_API_KEY`, such as `dex_test_mock`. The code then prints the answers above. ## Make your first call Send the same request to `https://api.thinqit.ai` with your key. The curl tab sends `request.json` from above. The Python and TypeScript tabs build the same request with the SDK helpers. Until the SDKs are on PyPI and npm, install them from [Downloads](/docs/reference/sdks/#downloads). ```bash tab="curl" curl https://api.thinqit.ai/v1/decide \ -H "authorization: Bearer $DEX_API_KEY" \ -H "content-type: application/json" \ -d @request.json ``` ```python tab="Python" # pip install thinqit-dex (coming soon; until then see Downloads on the SDKs page) from thinqit_dex import Client, pick, rate, check client = Client() # reads DEX_API_KEY decision = client.decide( state={ "ticket": { "channel": "email", "subject": "Charged twice, still no answer", "body": "This is the third time I am writing. You charged my card twice " "for order 4471 and nobody replies. Fix it today or I cancel my subscription.", } }, questions={ "team": pick( "Which team should handle {{ticket.body}}?", { "billing": "Payments, refunds and invoices", "technical": "Bugs, outages and sign-in problems", "shipping": "Delivery and returns", "other": None, }, min_confidence=0.5, ), "urgency": rate( "How fast must we reply to {{ticket.body}}?", ["Can wait", "This week", "Within two working days", "Today"], ), "angry": check("Is the writer of {{ticket.body}} angry?", min_confidence=0.6), }, ) team = decision.pick("team") print(team.abstained, team.choice, team.probabilities) print(decision.rate("urgency").rating) print(decision.check("angry").probability) ``` ```ts tab="TypeScript" // npm i @thinqit/dex (coming soon; until then see Downloads on the SDKs page) import { Client, pick, rate, check } from "@thinqit/dex"; const client = new Client(); // reads DEX_API_KEY const decision = await client.decide({ state: { ticket: { channel: "email", subject: "Charged twice, still no answer", body: "This is the third time I am writing. You charged my card twice for order 4471 and nobody replies. Fix it today or I cancel my subscription.", }, }, questions: { team: pick( "Which team should handle {{ticket.body}}?", { billing: "Payments, refunds and invoices", technical: "Bugs, outages and sign-in problems", shipping: "Delivery and returns", other: null, }, { min_confidence: 0.5 }, ), urgency: rate("How fast must we reply to {{ticket.body}}?", [ "Can wait", "This week", "Within two working days", "Today", ]), angry: check("Is the writer of {{ticket.body}} angry?", { min_confidence: 0.6 }), }, }); const { team, urgency, angry } = decision.answers; console.log(team.abstained, team.choice, team.probabilities); console.log(urgency.rating); console.log(angry.probability); ``` ## Read the answer Illustrative values: ```json { "id": "req_01M5D0000000000000000000AB", "object": "decision", "created": 1792497700, "model": "dex-1.0.0", "served_by": "gpu", "calibration": "cal-20261015-1", "answers": { "team": { "type": "pick", "choice": "billing", "probabilities": { "billing": 0.9112, "technical": 0.0301, "shipping": 0.0204, "other": 0.0383 }, "confidence": 0.8729, "abstained": false }, "urgency": { "type": "rate", "rating": 2.699, "levels": ["Can wait", "This week", "Within two working days", "Today"], "probabilities": [0.012, 0.061, 0.143, 0.784], "confidence": 0.5761, "abstained": false }, "angry": { "type": "check", "probability": 0.9421, "confidence": 0.8842, "abstained": false } }, "usage": { "input_tokens": 118, "state_tokens": 52, "question_tokens": 66, "charge_micro_cents": 590, "unit_price_micro_cents": 5, "tier": "t1" } } ``` ### The answers | Field | What it tells you | | --- | --- | | `team.choice` | The most probable option: `billing`. | | `team.probabilities` | One probability per option, in the order you sent them. They sum to exactly 1. | | `team.confidence` | For a pick, the gap between the two largest probabilities: 0.9112 minus 0.0383 is 0.8729. | | `team.abstained` | `false`, because 0.8729 is not below your `min_confidence` of 0.5. | | `urgency.rating` | The probability-weighted level index, from 0 (`Can wait`) to 3 (`Today`). 2.699 sits between `Within two working days` and `Today`, close to `Today`. | | `urgency.probabilities` | One probability per level, aligned with `levels`. | | `urgency.confidence` | How concentrated the probabilities are around the rating. 1 means all on one level. | | `urgency.abstained` | Always `false` here, because this question has no `min_confidence`. | | `angry.probability` | The probability that the answer is yes: 0.9421. | | `angry.confidence` | For a check, the distance from 50/50: two times 0.9421 minus 1 is 0.8842. | Confidence says how concentrated an answer is. It is not a promise that the answer is right. See [Confidence](/docs/concepts/confidence/). ### The envelope | Field | What it tells you | | --- | --- | | `id` | The request id. The same value is in the `x-request-id` header. Quote it to support. | | `model` | The exact version that answered. You asked for the alias `dex-1`, and the response names the version it resolved to. See [Models and versions](/docs/concepts/models/). | | `served_by` | `gpu` for our own engine, `fallback` for the hosted fallback. See [Fallback and served_by](/docs/concepts/fallback/). | | `calibration` | The calibration version applied to the probabilities. Each one has a public report on the [calibration page](/docs/calibration/). | | `usage.input_tokens` | The billed tokens: `state_tokens` (52) plus `question_tokens` (66). Output is free. | | `usage.charge_micro_cents` | What was debited: `input_tokens` times `unit_price_micro_cents`, here 118 times 5, is 590 micro-cents, which is EUR 0.0000059. One euro is 100,000,000 micro-cents. Test keys are charged 0. See [Tokens and billing](/docs/reference/billing/). | ## Act on it Use an answer only when it did not abstain. When `team` abstains, Dex is telling you that two teams are close, so send the ticket to a person instead of guessing. `route_to_person` and the other functions below stand for your own code. ```bash tab="curl" curl -s https://api.thinqit.ai/v1/decide \ -H "authorization: Bearer $DEX_API_KEY" \ -H "content-type: application/json" \ -d @request.json > response.json # Prints "person" when the team answer abstained, else the team jq -r 'if .answers.team.abstained then "person" else .answers.team.choice end' response.json ``` ```python tab="Python" team = decision.pick("team") if team.abstained: route_to_person(ticket) # not sure enough: a person decides else: route_to_team(ticket, team.choice) if decision.rate("urgency").rating >= 2.5: set_priority(ticket, "high") angry = decision.check("angry") if not angry.abstained and angry.probability >= 0.5: add_tag(ticket, "angry") ``` ```ts tab="TypeScript" const { team, urgency, angry } = decision.answers; if (team.abstained) { routeToPerson(ticket); // not sure enough: a person decides } else { routeToTeam(ticket, team.choice); } if (urgency.rating >= 2.5) setPriority(ticket, "high"); if (!angry.abstained && angry.probability >= 0.5) addTag(ticket, "angry"); ``` If a call fails, the body has an error `type` and `code`. See [Errors](/docs/reference/errors/#status-codes) for which ones to retry. ## Next steps - Learn the three [question types](/docs/concepts/questions/) and how to write good options and levels. - Choose a threshold with [Abstention](/docs/concepts/abstention/). - Read the [Known limits](/docs/concepts/known-limits/) before you automate: instructions hidden in the state, sarcasm, date arithmetic and Dutch criteria. - Pin a version for tests with [Determinism](/docs/concepts/determinism/). - Retry safely with [Idempotency](/docs/reference/idempotency/). - Check the [limits](/docs/reference/limits/) and [rate limits](/docs/reference/rate-limits/). - Read the full schema in the [API reference for decide](/docs/api/decide/). --- # State > The material every question is about, sent once per request as a string or as JSON, with its limits and billing. The state is the material you want decisions about: a support ticket, a forum post, a lead, a model's output. Every request has exactly one state, and all of its questions share it. ## String or JSON The `state` field takes one of three shapes: | Shape | Use it when | Example | | --- | --- | --- | | A string | The material is plain text, such as a message body. | `"This is the third time I am writing."` | | A JSON object | The material has named fields, such as a subject, a body and a channel. It needs at least one field. | `{ "ticket": { "subject": "...", "body": "..." } }` | | A JSON array | The material is a list, such as the messages in a thread. It needs at least one item. | `[ { "from": "customer", "text": "..." } ]` | A string is the simplest choice. JSON lets you do two more things: - Keep context the model can read, such as the channel or how long someone has been a customer. - Point a question at one field with a [field reference](/docs/concepts/field-references/), such as `{{ticket.body}}`. Field references need a JSON state: with a string state they give `422 state_not_json`. Here is the state of the Dutch support example from the contract. The questions point at `{{bericht.tekst}}`, and the model can also read the channel, the subject and the customer fields: ```json { "bericht": { "kanaal": "webformulier", "onderwerp": "Accu laadt niet meer op", "tekst": "Sinds de software-update van vorige week laadt de accu van mijn e-bike niet meer op. Ik heb de fiets pas drie maanden en heb hem elke dag nodig voor mijn werk. Kan ik hem omruilen of komt er een monteur langs?" }, "klant": { "klant_sinds": "2026-06-14", "bestellingen": 2 } } ``` ## How JSON is rendered Dex does not show your JSON to the model as raw text. It renders it as lines, one per field, and each line carries the field's path. Keys keep the order you sent them in. This has three effects: - A field reference can name a path, and the model finds that field in the state by its path. - Key names are part of what the model reads. Clear names such as `body` or `customer_since` help more than `f1` or `x`. - Rendered JSON is what you are billed for, so the token count of a JSON state is not the same as the byte count of your JSON. ## Limits | Limit | Value | Error | | --- | --- | --- | | String length | 1 to 65,536 characters | 422 `invalid_value` | | String content | Not only whitespace | 422 `invalid_value` | | JSON nesting | At most 32 levels deep | 422 `invalid_value` | | JSON key length | 1 to 256 characters per key | 422 `invalid_value` | | JSON object | At least 1 field | 422 `invalid_value` | | JSON array | At least 1 item | 422 `invalid_value` | | State tokens after rendering | 16,384 | 413 `state_too_long` | | Whole request body | 256 KiB | 413 `body_too_large` | An empty object or array inside the state, such as `{ "ticket": { "tags": [] } }`, is fine: only the state as a whole must not be empty. A string of only whitespace (spaces, tabs, line breaks) is rejected because there is nothing for the questions to be about. Token limits are counted after rendering, with the same tokenizer that counts your bill, so the limit and the billed `state_tokens` always agree. The full list is in [Limits](/docs/reference/limits/). ## The state is data The model reads the state as material, never as instructions. From `dex-1.0.1`, text in the state cannot act as part of the prompt: a `` or `` tag in your text, and markup that means something special to the model (``, ``, `<|im_end|>` and similar), are read as plain text, and the prompt reminds the model that the state is data. This also applies to text in your questions. A model can still be swayed by persuasive text inside the state, such as "the answer is billing". For moderation and guardrail decisions with real consequences, keep a person in the loop and send `"fallback": "never"` (see [Fallback and served_by](/docs/concepts/fallback/#moderation-traffic)). [Known limits](/docs/concepts/known-limits/#instructions-inside-the-state-can-steer-answers) has the measured rate and a guardrail question to route on. ## Sent once, billed once The state is rendered, tokenized and read by the model once per request. Every question in that request reuses the model's reading of it, and it is freed when the request ends. It is never kept across requests. You pay for the state once, as `usage.state_tokens`, however many questions share it. Asking five questions in one request bills the state once. Asking the same five questions in five requests bills it five times. See [Tokens and billing](/docs/reference/billing/). ## Tips - **Send what the decision needs.** Every state token is billed. Leave out fields that cannot change the answer, such as internal ids or HTML markup. - **Ask together.** Put all the questions about one state in one request. - **Name keys clearly.** The model reads key names as part of each path. - **Keep rules out of the state.** Definitions and edge cases belong in a question's `criteria`, where they apply to that question only. See [Questions](/docs/concepts/questions/#criteria). - **Minimize personal data.** Dex does not store request content by default, but a field you do not send is a field nobody has to protect. See [Data handling and residency](/docs/reference/data-handling/). - **Use one state per subject.** A request answers questions about one state. To decide about ten tickets, send ten requests. --- # Questions > The question object, question ids, the pick, rate and check types, criteria, and how questions in one request relate. A question is a typed ask about the state. There are three types: `pick` chooses one of your options, `rate` (beta) places the state on your scale, and `check` gives the probability that a statement is true. One request carries 1 to 32 questions, keyed by ids you choose. ## The question object | Field | Types | Required | Rule | | --- | --- | --- | --- | | `type` | all | yes | `"pick"`, `"rate"` or `"check"`. | | `instructions` | all | yes | What to decide, 1 to 4,000 characters, not only whitespace. May contain [field references](/docs/concepts/field-references/). | | `criteria` | all | no | Extra rules or definitions. See [Criteria](#criteria). | | `options` | pick | yes, for pick | Label to description. Rejected on other types. | | `levels` | rate | yes, for rate | Ordered level names. Rejected on other types. | | `min_confidence` | all | no | 0 to 1. Enables [abstention](/docs/concepts/abstention/). | A field that is not in this table is rejected with `422 unknown_field` (or `422 field_not_allowed` for a field of another question type), so a typo such as `instruction` fails loudly instead of being ignored. The same holds for the top level of the request, where an unknown field gives `422 unknown_field`. ## Question ids The keys of `questions` are your question ids. An id matches `^[A-Za-z][A-Za-z0-9_-]{0,63}$`: it starts with a letter, then has up to 63 more letters, digits, underscores or hyphens. Anything else gives `422 invalid_question_id`. Ids are for your code only. They never reach the model, so an id such as `angry` tells the model nothing. Put everything the model needs in `instructions`, `criteria`, `options` or `levels`. Answers come back under the same ids, in the order you sent the questions. ## pick A `pick` chooses one option from a set you define. ```json { "type": "pick", "instructions": "Which team should handle {{ticket.body}}?", "options": { "billing": "Payments, refunds and invoices", "technical": "Bugs, outages and sign-in problems", "shipping": "Delivery and returns", "other": null }, "min_confidence": 0.5 } ``` `options` maps each label to a description or `null`: - 1 to 255 options. More gives `422 too_many_options`. - A label is 1 to 100 characters, with no leading or trailing whitespace. - Labels must differ by more than case or Unicode normalization. `Billing` and `billing`, or two labels that differ only in how an accented letter is encoded, give `422 duplicate_label`: the model cannot tell them apart. - A description is `null` or 1 to 1,000 characters, and not only whitespace. Use `null` when the label says it all, as `other` does above. Add a description when a label could be read two ways. **Order matters.** Options are shown to the model in the order you send them, and a tie between probabilities goes to the earlier option. Sending the same options in a different order is a different request and may give different probabilities. Python dicts keep the order of every key. JavaScript objects, and so `JSON.parse`, do not: they list integer-like keys such as `"1"`, `"2"` and `"10"` first, in numeric order, whatever order your text used. A pick with labels `"10"`, `"2"`, `"1"` that goes through a plain JavaScript object reaches Dex as `"1"`, `"2"`, `"10"`, which is a different request. - **TypeScript SDK.** Pass labels to `pick()` in the order you want. For an object you built another way, mark its order with `withKeyOrder(obj, keys)` before `decide()`. On the way back, read a pick answer's `probabilities` with `orderedEntries()` to get option order. A request you read from a file keeps its order with `parseRequest(text)`. - **CLI and VS Code extension.** They read request files in file order and send them in that order. - **Your own code.** If your labels look like integers, build the JSON text yourself or use a parser that keeps order. **Include a way out.** If the state might fit none of your options, add one such as `other` or `none`. Otherwise the model must choose among options that do not fit. **More than 255 options.** Split the choice into stages: first pick a department, then send a second request that picks a queue within it. A pick answer has: | Field | Meaning | | --- | --- | | `choice` | The most probable label. Ties go to the earlier option. | | `probabilities` | Label to probability, in option order, summing to exactly 1. | | `confidence` | The gap between the two largest probabilities. See [Confidence](/docs/concepts/confidence/). | | `abstained` | `true` when `confidence` is below `min_confidence`. | On the fallback path, a pick with more than 20 options gets exact probabilities only for the 20 most likely options. See [Fallback and served_by](/docs/concepts/fallback/#what-differs-on-the-fallback). ## rate A `rate` places the state on an ordered scale. > **Beta.** `rate` works and is part of the API, but its measured quality misses our bars. On our test set, dex-1.0.1 picks the reference level on 63.7% of `rate` questions (the bar is 68%), and its calibration error is 0.138 (the bar is 0.05). Treat its probabilities as a rough guide, set thresholds from your own data, and set `min_confidence` so an unsure rating abstains. See [Quality bars](/docs/concepts/calibration/#quality-bars). ```json { "type": "rate", "instructions": "How fast must we reply to {{ticket.body}}?", "levels": ["Can wait", "This week", "Within two working days", "Today"] } ``` `levels` is a list of 2 to 10 unique strings, each 1 to 200 characters, ordered from lowest (index 0) to highest. Anything else gives `422 invalid_levels`, and so do two levels that differ only in case or Unicode normalization, such as `Low` and `low`. A level follows the label rule: no whitespace at either end and no line breaks, or `422 invalid_value` with the level in `param`, such as `questions.spoed.levels.2`. Write levels that form a real scale, because the rating treats their order as meaningful. A rate answer has: | Field | Meaning | | --- | --- | | `rating` | The probability-weighted level index, from 0 to the number of levels minus 1. | | `levels` | The levels you sent, in order. | | `probabilities` | One probability per level, aligned with `levels`, summing to exactly 1. | | `confidence` | 1 when all probability sits on one level, 0 when it splits evenly between the two extreme levels. | | `abstained` | `true` when `confidence` is below `min_confidence`. | `rating` is not rounded to a level. It is an average: it tells you where the probability centers, and `confidence` tells you how spread out it is. Compare the rating with thresholds of your own, round it when you need one level, or read `probabilities` when you need the whole distribution. ## check A `check` gives the probability that a statement about the state is true. ```json { "type": "check", "instructions": "Is the writer of {{ticket.body}} angry?", "min_confidence": 0.6 } ``` A check has no options or levels. Its answer has `probability` (the probability of yes), `confidence` and `abstained`. Phrase the statement so that yes is the case you act on. ## Criteria `criteria` adds rules and definitions the model must apply to one question. It takes: - a string of 1 to 4,000 characters, or - a list of 1 to 20 strings, each 1 to 500 characters, or - `null`, which is the same as leaving it out. A criteria string of only whitespace gives `422 invalid_value`, as does such an item in the list. Criteria may contain field references. Use them for edge cases and house rules. The English moderation example in the contract uses two: ```json { "type": "pick", "instructions": "Which community rule does {{post.text}} break, if any?", "criteria": [ "A threat to expose where someone lives counts as harassment even when it is conditional.", "Sharing your own phone number in a sale listing is allowed." ], "options": { "none": "Breaks no rule", "harassment": "Threats, intimidation or exposing someone's personal details", "spam": "Repeated or unsolicited promotion", "self_harm": "Encourages or describes self-harm", "hate": "Attacks people for a protected characteristic" }, "min_confidence": 0.6 } ``` Criteria are billed as question tokens, once per question that carries them. ## Questions in one request All questions share the state, which is read and billed once. Each question sees the fixed prompt prefix, the state and itself, not the text of the other questions. v1 does not promise question isolation. Adding, removing or reordering another question in the same request can move an answer slightly, because the questions are computed together. The same request bytes always give the same answers. If an answer must not depend on the other questions, send that question in a request of its own. See [Determinism](/docs/concepts/determinism/#what-v1-does-not-promise). This also means Dex does not make answers consistent with each other. Two differently phrased questions about the same fact can disagree. When exactly one of several outcomes must hold, ask one `pick` instead of several `check`s. Two checks such as "Is this about billing?" and "Is this about shipping?" can both come back above 0.5. A single `pick` with `billing`, `shipping` and `other` always returns exactly one choice, with probabilities that sum to 1. Some questions are weaker today: sarcasm read literally, date and amount arithmetic, and `criteria` written in Dutch. See [Known limits](/docs/concepts/known-limits/) for the numbers and how to work around each. --- # Field references > Point a question at one field of a JSON state with the path syntax, without paying for that field twice. When the state is JSON, `instructions` and `criteria` can point at a field with `{{path}}`. The model reads that field where it sits in the state, and you do not pay for it twice. ## Syntax A path is a dot-separated chain of keys, with array indexes in square brackets: | Reference | Points at | | --- | --- | | `{{ticket.body}}` | The `body` key inside `ticket` | | `{{bericht.tekst}}` | The `tekst` key inside `bericht` | | `{{ticket.messages[0].text}}` | The `text` key of the first item in `ticket.messages` | Array indexes start at 0. ### Keys with other characters Keys made of letters, digits, underscores and hyphens can be written as they are. A key with any other character, such as a space or a dot, goes in square brackets with double quotes: ```text {{customer["full name"]}} ``` ## Pointer semantics Dex does not paste the field's value into the question. The rendered question names the path, and the rendered state shows every field with its path, so the model reads the field in place. Two things follow: - **You are not billed twice.** The field's value counts once, in `state_tokens`. The reference itself adds only the few question tokens of the path. - **Many questions can point at one field.** Three questions that each reference `{{ticket.body}}` still pay for the body once. ## Validation Every referenced path must exist in the state. The check runs before anything is charged. | Problem | Error | | --- | --- | | The path is not in the state | 422 `state_path_not_found` | | The state is a string, not JSON | 422 `state_not_json` | A missing path error names the question field that holds the reference in `param`, and the path in `message`: ```json { "error": { "type": "validation", "code": "state_path_not_found", "message": "Question 'spoed' refers to {{bericht.txt}}, which is not in the state.", "param": "questions.spoed.instructions", "request_id": "req_01M5CJXHG0M9S346Q3D25VT4F5" } } ``` ## Literal braces - `{{` without a matching `}}` is literal text. - `\{{` escapes a literal `{{` that would otherwise start a reference. In a JSON string the backslash itself must be escaped, so the text `\{{` is written `"\\{{"` in the request body: ```json { "instructions": "Does {{post.text}} contain template syntax such as \\{{name}}?" } ``` ## SDK helpers Until the SDKs are on PyPI and npm, install them from [Downloads](/docs/reference/sdks/#downloads). `ref` builds a reference from path parts and adds brackets and quotes where a key needs them. `escape_braces` (Python) and `escapeBraces` (TypeScript) turn every `{{` in a text into `\{{`. ```python tab="Python" from thinqit_dex import ref, escape_braces ref("ticket", "messages", 0, "text") # "{{ticket.messages[0].text}}" ref("customer", "full name") # '{{customer["full name"]}}' escape_braces("a {{b}}") # "a \\{{b}}" ``` ```ts tab="TypeScript" import { ref, escapeBraces } from "@thinqit/dex"; ref("ticket", "messages", 0, "text"); // "{{ticket.messages[0].text}}" ref("customer", "full name"); // '{{customer["full name"]}}' escapeBraces("a {{b}}"); // "a \\{{b}}" ``` --- # Confidence > How the confidence value of each answer type is computed, what it does not promise, and how numbers are rounded. Every answer has a `confidence` between 0 and 1. It says how concentrated the answer's probability distribution is: 1 when all probability sits on one answer, lower as it spreads out. You use it to decide when to trust an answer, usually through [abstention](/docs/concepts/abstention/). ## From logits to probabilities For each question, the model produces one raw score (a logit) per possible answer: one per option for a `pick`, one per level for a `rate`, and a yes and a no score for a `check`. Dex divides these scores by a temperature `T` for the question type, then turns them into probabilities. The temperatures come from the model version's [calibration](/docs/concepts/calibration/), so there is one `T` each for pick, rate and check. The formulas below use `z` for the logits and `p` for the probabilities. ## pick The probabilities are a softmax over the options. The choice is the most probable option. Confidence is the gap between the two largest probabilities. ```text p_i = softmax(z_i / T_pick) over the n options choice = argmax p ties go to the earlier option confidence = p_(1) - p_(2) largest minus second largest confidence = 1 when there is only one option ``` In the Dutch support example in the contract, `garantie` has 0.6423 and `reparatie` has 0.3287, so confidence is 0.6423 minus 0.3287, which is 0.3136. Two options are close, and confidence is low. ## rate The probabilities are a softmax over the levels. The rating is their probability-weighted index. Confidence measures how tightly the probability clusters around the rating, relative to the widest possible spread. ```text p_j = softmax(z_j / T_rate) over the k levels rating = sum_j j * p_j sigma = sqrt(sum_j p_j * (j - rating)^2) confidence = 1 - sigma / ((k - 1) / 2) ``` Confidence is 1 when all probability sits on one level. It is 0 when the probability splits evenly between the lowest and the highest level, the widest spread possible. ## check The probability of yes is a sigmoid of the gap between the yes and no scores. Confidence is the distance from 50/50, scaled to 0 to 1. ```text probability = sigmoid((z_yes - z_no) / T_check) confidence = |2 * probability - 1| ``` This is the same gap as a `pick` with two options. A probability of 0.5 has confidence 0, and a probability of 0.75 has confidence 0.5. It helps to translate a threshold back into probabilities. A check reaches confidence 0.6 only when the probability is at least 0.8 or at most 0.2. ## What confidence is not - **Not a correctness guarantee.** A confident answer can be wrong. Confidence only describes the shape of the distribution. [Calibration](/docs/concepts/calibration/) is what makes the probabilities mean what they say. - **Not one scale across types.** Each type has its own formula, so 0.5 on a `rate` does not mean the same as 0.5 on a `pick`. Choose `min_confidence` per question, not one number for all. - **Not a probability.** For a check, `probability` is the probability of yes, and `confidence` is derived from it. Use `probability` when you need the chance that something holds. ## Rounding Dex rounds numbers the same way every time, so the same inputs always give the same bytes. - **Probabilities** are rounded to 4 decimals with the largest-remainder method, so each distribution sums to exactly 1.0000. Equal remainders go to the earlier option or level. - **`rating`, `confidence` and a check's `probability`** are computed from the unrounded probabilities, then rounded half to even to 4 decimals. - **`choice` and `abstained`** are decided on the unrounded values. - **Numbers** are written in plain decimal notation, never with an exponent. Because confidence comes from unrounded values, recomputing it from the rounded numbers can differ in the last digit. In the Dutch support example, the check has `probability` 0.8732 and `confidence` 0.7465, while two times 0.8732 minus 1 gives 0.7464. --- # Abstention > Let an unsure answer say so with min_confidence, and route it to a person instead of acting on a guess. Set `min_confidence` on a question, and an answer whose confidence falls below it comes back with `abstained: true`. Your code can then send the case to a person, ask again with more information, or log it, instead of acting on a guess. ## How it works `min_confidence` is an optional number from 0 to 1 on any question. A value outside that range gives `422 invalid_min_confidence`. | Setting | `abstained` | | --- | --- | | No `min_confidence` | Always `false` | | `confidence` below `min_confidence` | `true` | | `confidence` equal to or above `min_confidence` | `false` | Each question has its own threshold, so in one request a team choice can abstain while an urgency rating does not. ## What an abstained answer contains An abstained answer still returns every typed field: `choice` and `probabilities` for a pick, `rating` for a rate, `probability` for a check. You can log and inspect them. Do not act on them. Treat code that uses an abstained answer's `choice`, `rating` or `probability` as a bug. The SDK answer types list `abstained` first to make that hard to miss. ## Deterministic and free - **Free.** Abstention costs nothing extra. The request is billed the same with or without `min_confidence`. - **Deterministic.** Dex decides abstention in its gateway, from the unrounded confidence. The same request to the same GPU version always abstains, or does not, in the same way. See [Determinism](/docs/concepts/determinism/). ## Example: the Dutch e-bike ticket A customer writes that their e-bike battery stopped charging after a software update, and asks whether they can exchange the bike or whether a mechanic can come by. The request asks which department should handle it (`afdeling`, with `min_confidence` 0.5), how urgent it is (`spoed`), and whether the customer asks for an exchange (`wil_omruilen`, with `min_confidence` 0.6): ```json { "model": "dex-1", "state": { "bericht": { "kanaal": "webformulier", "onderwerp": "Accu laadt niet meer op", "tekst": "Sinds de software-update van vorige week laadt de accu van mijn e-bike niet meer op. Ik heb de fiets pas drie maanden en heb hem elke dag nodig voor mijn werk. Kan ik hem omruilen of komt er een monteur langs?" }, "klant": { "klant_sinds": "2026-06-14", "bestellingen": 2 } }, "questions": { "afdeling": { "type": "pick", "instructions": "Welke afdeling moet {{bericht.tekst}} oppakken?", "options": { "garantie": "Omruilen of terugbetalen binnen de garantietermijn", "reparatie": "Defecten, onderhoud en monteur aan huis", "bezorging": "Levering en track-and-trace", "overig": null }, "min_confidence": 0.5 }, "spoed": { "type": "rate", "instructions": "Hoe snel moeten we reageren op {{bericht.tekst}}?", "levels": ["Kan wachten", "Binnen een week", "Binnen twee werkdagen", "Vandaag"] }, "wil_omruilen": { "type": "check", "instructions": "Vraagt de klant in {{bericht.tekst}} om omruilen of vervanging?", "min_confidence": 0.6 } } } ``` The response from the contract example: ```json { "id": "req_01M5CJXHG0M9S346Q3D25VT4F5", "object": "decision", "created": 1792497600, "model": "dex-1.0.0", "served_by": "gpu", "calibration": "cal-20261015-1", "answers": { "afdeling": { "type": "pick", "choice": "garantie", "probabilities": { "garantie": 0.6423, "reparatie": 0.3287, "bezorging": 0.0102, "overig": 0.0188 }, "confidence": 0.3136, "abstained": true }, "spoed": { "type": "rate", "rating": 2.2957, "levels": ["Kan wachten", "Binnen een week", "Binnen twee werkdagen", "Vandaag"], "probabilities": [0.0195, 0.1067, 0.4325, 0.4413], "confidence": 0.5108, "abstained": false }, "wil_omruilen": { "type": "check", "probability": 0.8732, "confidence": 0.7465, "abstained": false } }, "usage": { "input_tokens": 273, "state_tokens": 142, "question_tokens": 131, "charge_micro_cents": 1365, "unit_price_micro_cents": 5, "tier": "t1" } } ``` Reading it: - **`afdeling` abstains.** The message fits both warranty (`garantie`, 0.6423) and repair (`reparatie`, 0.3287). The gap between them, 0.3136, is below the 0.5 threshold. `choice` is still `garantie`, but your code should send this ticket to a person to choose the department. - **`spoed` does not abstain.** It has no `min_confidence`. Its rating of 2.2957 sits between "within two working days" and "today". - **`wil_omruilen` does not abstain.** Confidence 0.7465 is above 0.6, and the customer asks for an exchange with probability 0.8732. So the ticket goes to a person for routing, with its urgency and the exchange request already filled in. ## Patterns - **Route to a person.** The most common pattern. Put abstained cases in a review queue, with the answer's probabilities attached so the reviewer sees what was close. - **Ask again with more information.** On a GPU version, the same request always gives the same answer, so repeating it does not help. Send a new request when the state has changed, for example after the customer replies, or ask a narrower question. - **Use a safe default.** For an agent's proposed tool call, an abstained "allow, confirm or deny" pick can default to asking the user to confirm. - **Log it.** Count abstentions per question. A high rate on one question often means two options overlap and need clearer descriptions or `criteria`. Until the SDKs are on PyPI and npm, install them from [Downloads](/docs/reference/sdks/#downloads). ```python tab="Python" afdeling = decision.pick("afdeling") if afdeling.abstained: send_to_review_queue(ticket, afdeling.probabilities) else: assign(ticket, afdeling.choice) ``` ```ts tab="TypeScript" const { afdeling } = decision.answers; if (afdeling.abstained) { sendToReviewQueue(ticket, afdeling.probabilities); } else { assign(ticket, afdeling.choice); } ``` ## Choosing a threshold - **Start from the formula.** `confidence` means something different per type. For a `check`, `min_confidence` 0.6 abstains unless the probability is at least 0.8 or at most 0.2. For a `pick`, 0.5 abstains unless the top option leads the second by at least 0.5. See [Confidence](/docs/concepts/confidence/). - **Measure on your own traffic.** Run a sample of real cases with a test key, which is free up to 250,000 tokens a day, and look at how many answers abstain at a few thresholds and how often the non-abstained answers match what your team would decide. - **Weigh the cost of a mistake.** A threshold is a trade between how many cases a person handles and how many wrong answers get through. Set it higher where a wrong answer is expensive. --- # Determinism > The same request bytes to the same GPU model version return the same answers, byte for byte. Scope, limits, method and checks. On a GPU model version, the same request always returns the same answers, byte for byte. You can snapshot-test against Dex, replay a request to audit a past decision, and retry without getting a different result. ## The guarantee On the GPU path, identical request bytes sent to the same exact model version return byte-identical `answers`, `model`, `calibration` and `usage`. This holds: - whether you name the exact version (`dex-1.0.0`) or the alias (`dex-1`), as long as the alias resolves to that version; - across time and across load; - whatever other requests are on the GPU at the same moment. Other requests never change your result. Send the same bytes and you get the same bytes back. Keep your request body fixed in your tests, including the order of keys: reordering options, or the objects of a JSON state, is a different request. ## What v1 does not promise - **Question isolation.** The questions of one request are computed together after the shared state. Adding, removing or reordering another question in the same request can move an answer slightly, because the GPU rounds differently when the packed input gets longer or shorter. The same request bytes still give the same answers every time. Computing each question exactly as if it were asked alone is on the roadmap. - **Equivalent spellings.** The guarantee is for identical request bytes. Bodies that differ only in whitespace or in Unicode normalization render the same text for the model, so their probabilities match in practice. Labels and levels are echoed exactly as you sent them, so a label in a different Unicode form comes back in that form. If you need an answer that cannot depend on the other questions, send that question in a request of its own. ## What differs between calls - `id` and `created` are new on every call. An [idempotent replay](/docs/reference/idempotency/) returns the original ones. - Response headers such as `x-request-id` and the rate limit headers change. ## Scope The guarantee covers GPU versions only. - **Fallback versions are excluded.** The hosted fallback is not bit-deterministic. Its responses say so: `served_by` is `fallback`, `model` is a `dex-fallback-*` version, and `GET /v1/models` lists it with `deterministic: false`. See [Fallback and served_by](/docs/concepts/fallback/). - **Pinning an exact version keeps you on the GPU.** A request that names an exact GPU version is never sent to the fallback. If no GPU can serve it, it gets `503`. - **`fallback: "never"` does the same for the alias.** The request stays on the GPU path and gets `503` when no GPU can serve it. - **The alias moves.** When a new version ships, `dex-1` resolves to it, and answers can change. The [changelog](/docs/changelog/) records every move. To hold answers fixed over time, pin an exact version. See [Models and versions](/docs/concepts/models/). ## How it is achieved GPU inference is usually not bit-reproducible: the order of floating-point additions can change with batch size, and that changes the last bits of the result. Dex removes each source of variation that other requests could bring in. 1. **One request per GPU pass, or batch-invariant kernels.** The engine serves one request per pass. When it batches requests, it uses kernels whose sums run in the same order no matter how many requests share the pass. Kernel settings are fixed when the engine is built, not tuned at run time. 2. **Shared state, packed questions.** The state is read once. The questions follow it in one packed input, and a mask keeps each question from reading the others. The packed length still affects rounding, which is why v1 does not promise question isolation. 3. **Fixed padding and order.** Inputs are padded to fixed sizes and laid out in a fixed order: prefix, state, then questions. The engine only ever runs a small set of shapes, all compiled in advance. 4. **No sampling.** Answers come from the model's scores in one pass over the input. There is no random number generator anywhere on the path. 5. **Pinned profile and golden gate.** An exact version fixes the weights, tokenizer, prompt template, the inference software and hardware configuration, and the calibration. When the GPU worker starts, it runs a suite of 500 golden requests for each version and compares a hash of the raw scores with the version's stored golden hash. On a mismatch, for example after an unplanned driver update, it does not serve that version and raises an alert. 6. **One post-processing implementation.** Temperature scaling, softmax, rounding and confidence run in one place, in the gateway, in 64-bit floating point. Fixed test vectors (known scores in, known bytes out) must pass before the gateway can deploy. ## How it is verified - **A determinism suite in CI, on the real GPU.** Every golden request runs alone, 20 times in a row, and mixed with other requests under load. Every response must be byte-identical each time. Shuffled question orders and single-question subsets are measured and reported for the isolation roadmap item. - **A production canary every 5 minutes.** A fixed synthetic request, with no customer data, runs against every live version and is compared with its stored bytes. A mismatch pages the on-call engineer and takes the affected inference node out of service. Results of both are published at launch. ## Using it - **Snapshot tests.** Pin an exact version, send fixed request bodies with a test key, and store the responses. Compare `answers` and `usage` on later runs. Test keys always use the GPU path, and they are free up to 250,000 tokens a day. - **Audits.** To explain a past decision, send the same request bytes to the same exact version. Store the request, the `model` and the `calibration` from the response. - **Safe retries.** With an [idempotency key](/docs/reference/idempotency/), a retried request is charged at most once and returns the original response. - **Upgrades.** When a new version ships, run your snapshots against it before you move your pin. Superseded versions stay available for at least 90 days. --- # Calibration > What calibrated probabilities mean, how each model version is calibrated and measured, the measured numbers, and the bars they are held to. Calibration is what makes a probability mean what it says. When Dex says 80%, it should be right about 80% of the time. Every model version is calibrated before it ships, and every response names the calibration version behind its numbers. ## What calibration means Take every answer where Dex gave its top choice a probability near 0.8. If the probabilities are calibrated, about 80% of those choices are correct. The same should hold at 0.6, at 0.95, and everywhere else. A model can rank options well and still be overconfident or underconfident. Calibration fixes the scale, so you can set thresholds on probabilities and reason about how often an answer above them is wrong. ## How a version is calibrated Dex uses temperature scaling, with one temperature per question type: `T_pick`, `T_rate` and `T_check` for each exact model version. - A temperature is one number that sharpens or softens a distribution. Raw scores are divided by it before they become probabilities (see [Confidence](/docs/concepts/confidence/#from-logits-to-probabilities)). - Temperature scaling never changes which answer is most likely. It changes only how sure the probabilities are. - Each temperature is fitted by minimizing negative log-likelihood on the dev split of our data, searching over temperatures from 0.25 to 8. - Fallback versions get their own three temperatures, fitted on the fallback model's own scores. The set of temperatures for a version has an id of the form `cal-YYYYMMDD-n`. Every response names it in `calibration`, and `GET /v1/models` lists it per version. ## Data and reference labels dex-1.0.1 was measured on our own decision set, Dex decision eval v1. - **The items.** 400 items, 200 in English and 200 in Dutch, with 1,224 questions across `pick`, `rate` and `check`. They cover seven kinds of work: support routing, moderation, lead intent, product categorisation, document triage, sentiment and stance, and agent guardrails. 50 items are hard cases: negation, literal instructions, numbers and dates, long noisy text, and prompt injection. - **Where they come from.** Every item was written for Dex: by AI models working from our briefs, with fictional people, companies and contact details. No third-party text or datasets were used. - **Reference labels.** Two frontier models from two different vendors label every question with a probability per answer. The reference is the average of the two, and its most likely answer is the reference label. - **No customer data.** Customer content is never used to train, tune or calibrate any model. - **Splits.** 100 items form the dev split, used to fit the temperatures. The other 300 form the test split, which is frozen and used only for the reported numbers. The test split is never used for fitting or for choosing a model. **Known limits of this set.** Say so before you rely on the numbers: - The items are synthetic, written by AI models of one family, and cleaner than real traffic. - The Dutch items have not yet been checked by a native speaker, and no person has settled the questions where the two reference models disagree. - The set is small, so the numbers for smaller groups, such as the `rate` questions, carry several points of uncertainty. - Most `pick` and `check` questions are clear-cut, so their calibration is measured mostly at high probabilities. ## Metrics All metrics are reported on the test split: overall, per language and per question type. **Expected calibration error (ECE).** Sort the answers by their predicted probability and split them into 15 bins with the same number of answers in each. In each bin, compare the average predicted probability with how often the answers were actually right. ECE is the average of those gaps across the bins. 0 is perfect. | Type | What is measured | | --- | --- | | `pick` | Top-label ECE: the top choice's probability against whether the top choice was right. | | `check` | Binary ECE: `probability` against the yes or no outcome. | | `rate` | Top-level ECE, plus the mean absolute error of `rating` against the reference level. | **Agreement.** How often Dex matches the reference label: | Type | Counts as agreement when | | --- | --- | | `pick` | `choice` equals the reference option. | | `check` | `probability` is 0.5 or more and the reference says yes, or below 0.5 and it says no. | | `rate` | The most probable level equals the reference level. | For `rate` we also report agreement of `rating` rounded to the nearest level. ## Quality bars Our bars: agreement of at least 68% overall and for each question type, at least 64% in each language, and ECE of at most 0.05 for each question type. dex-1.0.1 meets the agreement bars overall (82.3%) and in both languages (English 84.9%, Dutch 79.6%). Its ECE over all questions is 0.044. It misses two bars, and we publish the numbers as they are: - **ECE per type is above 0.05** for all three types: `pick` 0.048, `check` 0.052, `rate` 0.138. - **`rate` agreement is 63.7%**, below 68%. That is why `rate` is labelled beta. See [Questions](/docs/concepts/questions/#rate). What this means in practice: treat a probability as a good guide, not an exact rate, and for `rate` set your thresholds from your own data. ## Publication Each calibration version gets a report on the [calibration page](/docs/calibration/). For dex-1.0.1 it shows agreement overall, per language and per question type, ECE overall and per question type, the number of questions behind each, and a reliability curve per question type. ECE and curves per language, bootstrap bands, Brier scores and the fitted temperatures are not published yet. Because every response names its `calibration`, you can trace any answer to the numbers behind it. ## Measured results Model `dex-1.0.1`, calibration `cal-20260926-1`. Measured 2026-09-26 on Dex's production path in the EU. Evaluation set: Dex decision eval v1, test split: 300 items, 921 questions, English and Dutch; reference = average of gpt-5.3-chat and claude-opus-5 (they agree on 96.3%); temperatures fitted on the 100-item dev split. | Measurement | Value | | --- | --- | | Agreement with reference labels, English and Dutch (921 questions) | 82.3% | | Agreement, English (465 questions) | 84.9% | | Agreement, Dutch (456 questions) | 79.6% | | Agreement, pick (400 questions) | 83.0% | | Agreement, check (353 questions) | 90.4% | | Agreement, rate (beta) (168 questions) | 63.7% | | Calibration error (ECE), all questions (921 questions) | 0.044 | | Calibration error (ECE), pick (400 questions) | 0.048 | | Calibration error (ECE), check (353 questions) | 0.052 | | Calibration error (ECE), rate (beta) (168 questions) | 0.138 | | Latency p50, GPU path, end to end | Published at launch | | Latency p95, GPU path, end to end | Published at launch | | Determinism canary mismatches | Published at launch | For `rate`, agreement counts the most probable level. Counted on `rating` rounded to the nearest level, it is 58.3%. The rating is off by 0.591 levels on average. Percentages are rounded to one decimal and ECE to three, and a value exactly halfway is rounded down. --- # Models and versions > The model alias, exact versions and what they freeze, the fallback versions, the version lifecycle and the models endpoint. You choose a model with the `model` field of a request. It takes an alias, which moves to newer versions over time, or an exact version, which never changes. Every response names the exact version that answered. ## Alias and exact version | Form | Example | What it does | | --- | --- | --- | | Alias | `dex-1` | Resolves to the newest active exact version in major version 1. It moves when a new version ships, and the [changelog](/docs/changelog/) records each move. This is the default when you leave `model` out. | | Exact version | `dex-1.0.0` | A frozen serving profile, named `dex-MAJOR.MINOR.PATCH`. The same request to it always gives the same answers. | The `model` field of every response is the exact version that answered, even when you sent the alias. A name that does not exist gives `404 model_not_found`. A version that has been retired gives `404 model_retired`. ## What an exact version freezes An exact version fixes everything that could change an answer: - the model weights, the tokenizer and the quantization recipe; - the prompt template and the codes used to read answers; - the inference software build with its pinned dependencies, and the hardware configuration; - the [calibration](/docs/concepts/calibration/) version; - a golden hash: a fingerprint of the engine's raw outputs on a fixed suite of 500 requests. A change to any of these ships as a new version. A new calibration alone is a new PATCH version. A new tokenizer is a new major version, so every version in major version 1 counts tokens the same way. ## Versions | Version | Released | What changed | | --- | --- | --- | | `dex-1.0.0` | 25 September 2026 | The first version, calibration `cal-20260925-1`. | | `dex-1.0.1` | 26 September 2026 | The alias `dex-1` points here. Your text can no longer act as prompt structure: a `` inside the state, or markup such as `` and `` anywhere in a request, is read as plain text. Same model and engine as `dex-1.0.0`; calibration `cal-20260926-1`. | Requests without such text get the same decisions on both versions in nearly every case; the probabilities differ slightly, because `dex-1.0.1` reads one more line after the state and has its own calibration. See the [changelog](/docs/changelog/) and the [calibration reports](/docs/calibration/). ## Fallback versions When the GPU path cannot answer an alias request, a hosted fallback may answer instead. It has versions of its own: | Version | Upstream model | | --- | --- | | `dex-fallback-1.0.0` | Azure OpenAI gpt-4.1-mini (2025-04-14), Data Zone Standard, EU | | `dex-fallback-lite-1.0.0` | Azure OpenAI gpt-4o-mini (2024-07-18), Data Zone Standard, EU | Each is calibrated separately and is not deterministic. You cannot request a fallback version directly; Dex chooses between them. Both upstream models retire on 14 April 2027, which `GET /v1/models` shows as their `retires_at`. See [Fallback and served_by](/docs/concepts/fallback/). ## Lifecycle A version's `status` is `active`, `deprecated` or `retired`. - When a new version becomes the alias target, the version it replaces stays available for at least 90 days. - `GET /v1/models` shows each version's `retires_at`. - Account owners get an email 30 days and 7 days before a version retires. - After that, a request that names it gets `404 model_retired`. Within v1, the API changes only by adding things: new optional request fields, new response fields and new error codes. Write your code to ignore response fields it does not know. ## Which one to use - **Use the alias** to get improvements without changing code. Expect answers to change when the alias moves, and watch the changelog. - **Pin an exact version** when answers must not change, for example for snapshot tests, audits or regulated workflows. Plan the move to a new version within the 90-day window. A pinned version is never served by the fallback, so when its GPU capacity is unavailable you get `503` instead of an answer. See [Determinism](/docs/concepts/determinism/#scope). ## List models `GET /v1/models` lists the alias, the exact versions and the fallback versions. Any valid API key can call it. ```bash curl https://api.thinqit.ai/v1/models \ -H "authorization: Bearer $DEX_API_KEY" ``` Each entry looks like this alias entry from the contract example: ```json { "id": "dex-1", "object": "model", "kind": "alias", "target": "dex-1.0.0", "served_by": "gpu", "status": "active", "deterministic": true, "released_at": "2026-10-15T00:00:00Z", "retires_at": null, "calibration": "cal-20261015-1", "upstream": null, "limits": { "max_state_tokens": 16384, "max_question_tokens": 4096, "max_total_tokens": 16384, "max_questions": 32, "max_options": 255, "max_levels": 10, "exact_option_probabilities": 255 }, "quality": null } ``` | Field | Meaning | | --- | --- | | `kind` | `alias` or `version`. | | `target` | For an alias, the exact version it resolves to. | | `served_by` | `gpu` or `fallback`. | | `status` | `active`, `deprecated` or `retired`. | | `deterministic` | `true` when the same request always gives the same answers from this version. | | `released_at`, `retires_at` | When the version was released and when it retires (`null` when no date is set). | | `calibration` | The calibration version applied. | | `upstream` | For a fallback version, the hosted model and region that serve it. | | `limits` | The request limits for this version. See [Limits](/docs/reference/limits/#model-limits). | | `quality` | Published agreement and ECE figures with a link to the calibration report, or `null`. | The full schema is in the [API reference for listModels](/docs/api/listModels/). --- # Fallback and served_by > When a hosted fallback in the EU answers instead of the GPU, how every response says so, and how to keep a request on the GPU. Dex answers from its own EU inference nodes: dedicated EU hardware operated by thinQit. When that node cannot answer an alias request in time, a hosted fallback on Azure OpenAI in the EU data zone can answer instead. Every response tells you which path answered in `served_by`, and you can keep any request on the GPU path. ## Why a fallback exists When those nodes are down, restarting, overloaded or in maintenance, the fallback keeps alias requests answered. Planned maintenance is announced 48 hours ahead on the status page at https://thinqit.ai/status/, and uses the fallback while it lasts. ## When a request goes to the fallback A request can go to the fallback only when all of these hold: - `model` is the alias `dex-1`, not an exact version; - `fallback` is `"allow"`, which is the default; - the key is a live key. Test keys and console playground calls always use the GPU path. It then goes to the fallback when the GPU path cannot answer it in time: - no ready and healthy GPU worker serves the version, or its queue is too long to answer in time; - the GPU worker fails the job, does not answer in time, or goes offline while the job runs. The fallback also has a spending limit. When it is closed or also fails, the request gets `503 no_capacity` or `503 upstream_unavailable` with a `retry-after` header, and any charge is refunded. ## served_by in every response Every response has three fields that tell you how it was answered: | Field | GPU path | Fallback path | | --- | --- | --- | | `served_by` | `gpu` | `fallback` | | `model` | An exact GPU version, such as `dex-1.0.0` | `dex-fallback-1.0.0` or `dex-fallback-lite-1.0.0` | | `calibration` | The GPU version's calibration | The fallback version's own calibration | `GET /v1/usage` counts `gpu_requests` and `fallback_requests` per time bucket, so you can see how much of your traffic each path served. ## Keeping a request on the GPU There are two ways: - Send `"fallback": "never"` with the alias. - Pin an exact GPU version, such as `dex-1.0.0`. Exact versions never use the fallback, whatever `fallback` says. The trade-off: when no GPU can serve the request, you get `503` with a `retry-after` header instead of a fallback answer. The English moderation example in the contract does both: ```json { "model": "dex-1.0.0", "fallback": "never", "state": { "post": { "board": "Bikes for sale", "title": "Selling my old road bike", "text": "Still available, cash only. Text me on +31 6 1234 5678. The next person who lowballs me will find their home address posted in this thread." } }, "questions": { "rule": { "type": "pick", "instructions": "Which community rule does {{post.text}} break, if any?", "criteria": [ "A threat to expose where someone lives counts as harassment even when it is conditional.", "Sharing your own phone number in a sale listing is allowed." ], "options": { "none": "Breaks no rule", "harassment": "Threats, intimidation or exposing someone's personal details", "spam": "Repeated or unsolicited promotion", "self_harm": "Encourages or describes self-harm", "hate": "Attacks people for a protected characteristic" }, "min_confidence": 0.6 }, "severity": { "type": "rate", "instructions": "How severe is the worst problem in {{post.text}}?", "levels": ["None", "Mild", "Serious", "Severe"] }, "has_phone_number": { "type": "check", "instructions": "Does {{post.text}} contain a phone number?" } } } ``` ## What differs on the fallback - **Not deterministic.** The same request can give different numbers on the fallback, and a retry can be answered by the other path. `GET /v1/models` lists fallback versions with `deterministic: false`. See [Determinism](/docs/concepts/determinism/#scope). - **Its own calibration.** Fallback versions have their own temperatures and their own calibration reports. - **More than 20 options.** On the fallback, a pick with more than 20 options gets exact probabilities only for the 20 most likely options. The remaining probability is spread evenly over the other options. The model's `limits.exact_option_probabilities` shows 20 for fallback versions. - **Content filters.** Azure OpenAI's default content filters block prompts with hate, sexual, violent or self-harm content above medium severity. If the filter blocks any question in a request, the fallback fails for the whole request, and it gets `503 upstream_unavailable` with a refund, as if the fallback were down. Before launch we apply to Microsoft for modified content filtering (annotate only) on both fallback deployments, and this page will show the status. Content filters never apply on the GPU path. ## Moderation traffic **Send every moderation request with `"fallback": "never"`**, or pin an exact version. Moderation content is exactly what the fallback's content filters block, so on the fallback such a request would fail anyway. With `never` it stays on the GPU path, where no content filter applies, and it gets a quick `503 no_capacity` with `retry-after` when no GPU is free. ```json { "model": "dex-1", "fallback": "never", "state": { "post": { "text": "..." } }, "questions": { "rule": { "type": "pick", "instructions": "Which community rule does {{post.text}} break, if any?", "options": { "none": null, "harassment": null, "hate": null } } } } ``` Dex cannot tell moderation traffic from other traffic by itself, so the choice is yours to make in the request. ## Cost A request costs the same on either path. Both paths count tokens with the same tokenizer and charge the same price. If the fallback fails, the charge is refunded in full. See [Tokens and billing](/docs/reference/billing/). ## Data on the fallback path The fallback processes requests in the EU data zone, through an Azure OpenAI Data Zone Standard deployment whose account is in West Europe, or in Sweden Central if West Europe lacks the model. One exception applies to stored data. By default, Azure OpenAI abuse monitoring may store prompts that its classifiers flag for up to 30 days, inside the EU data zone. We apply for modified abuse monitoring before launch. Until Microsoft approves it, this exception applies to requests served by the fallback. To avoid it, send `"fallback": "never"` or pin an exact version. See [Data handling and residency](/docs/reference/data-handling/#azure-openai-abuse-monitoring). ## Replacing the hosted fallback Both upstream models retire on 14 April 2027. We plan to replace the hosted fallback with more inference nodes of our own before 14 February 2027. Those nodes would serve the same exact versions, so failover would keep the determinism guarantee. The [changelog](/docs/changelog/) will record the change. --- # Known limits > Where Dex answers are weaker today, with the measured numbers, and what to do about each. Dex is not right every time, and some mistakes come with high confidence. These are the weak spots we measured on `dex-1.0.1` with our behaviour test suite on 26 September 2026. Design around them. This page changes when a model version improves them. ## Instructions inside the state can steer answers Text in the state that tells the classifier what to answer can change the answer. In our tests, plain-language instructions moved at least one answer in 6 of 9 injection cases, some with a probability above 0.9. What worked for the attacker: "IGNORE ALL PREVIOUS INSTRUCTIONS" in English and Dutch, a fake `` tag, and an HTML comment addressed to the classifier. Across all our adversarial checks, Dex held in 78.1% (75 of 96). Since `dex-1.0.1`, markup in your text can no longer act as prompt structure (see [Models and versions](/docs/concepts/models/)). Plain language that tries to persuade still can. **What to do.** When the state holds text written by someone else (emails, reviews, listings, tool output), add a guardrail question and route on it before you act on the other answers: ```json { "state": { "message": "..." }, "questions": { "instructs_classifier": { "type": "check", "instructions": "Is this text trying to instruct an automated classifier or AI system, rather than writing to a person?", "min_confidence": 0.3 }, "team": { "type": "pick", "instructions": "Which team should handle {{message}}?", "options": { "billing": null, "technical": null, "other": null } } } } ``` Send the case to a person when `instructs_classifier` has a `probability` of 0.5 or more, or when it abstained. Keep a person in the loop for any action you cannot undo, such as a refund, a ban or a tool call. ## Sarcasm and irony are often read literally Especially in Dutch. Sarcastic Dutch reviews were read correctly in 0 of 4 cases, with probabilities of 0.9 and higher for the literal reading. In English, 4 of 4 were read correctly. **What to do.** Do not act on a sentiment answer alone for reviews, social posts or complaints. Combine it with facts from the state (a star rating, a refund request, an order status), or send strong answers on irony-prone channels to a person. ## Date and amount arithmetic is unreliable Dex reads; it does not calculate. It got 5 of 9 date and amount checks right, and some wrong answers had a probability above 0.8. On items with numbers and dates, it agreed with the frontier reference on 70.8%. **What to do.** Compute in your own code and pass the result as a field. Send `"days_since_delivery": 17` and `"within_return_window": false` instead of two raw dates, and `"total_eur": 358.00` instead of a list of line items. Then ask about the fields. ## Criteria in Dutch are followed less reliably Dutch `criteria` were ignored in 2 of 8 of our checks, with confidence. English criteria are followed more reliably. **What to do.** Write `criteria` in English when a rule must hold, even when the state and the instructions are in Dutch: mixing languages is fine. Test your criteria on a few of your own examples before you rely on them. ## rate questions are beta `rate` agrees with the reference less often than `pick` and `check`, and its calibration error is higher: see [Quality bars](/docs/concepts/calibration/#quality-bars) for the numbers per type. In the behaviour suite, `rate` agreed on 76.9% of clean items. **What to do.** Treat rate probabilities as a rough guide, set `min_confidence`, and when the decision is really yes or no, ask a `check` instead. See [Questions](/docs/concepts/questions/#rate). ## Also measured - **Borderline answers can flip.** Under harmless changes to the wording, 92.1% of decisions held. 59% of the flips started from an answer with a probability below 0.6, so a `min_confidence` catches many of them. See [Abstention](/docs/concepts/abstention/). - **Content moderation is the weakest domain** in the suite: 64.7% (17 items) agreed with the frontier reference. Send `fallback: "never"` for moderation (see [Fallback](/docs/concepts/fallback/)) and review removals. --- # Authentication and API keys > Send your API key as a bearer token, check which key you use, and validate the key format and checksum offline. Every public route takes an API key in the `authorization` header. Keys come from the console at https://thinqit.ai/console/: sign in with your email address and create a test or live key. Test keys are free. This page covers the header, the two kinds of key, scopes, how to check which key you are using, and the key format with its built-in checksum. ## Send the key ```bash curl https://api.thinqit.ai/v1/models \ -H "authorization: Bearer $DEX_API_KEY" ``` The SDKs and the CLI read the key from the `DEX_API_KEY` environment variable, or take it as an argument. Never put a key in source code, a URL or a log line. A missing key gives `401 missing_api_key`. A key that does not exist gives `401 invalid_api_key`. A revoked or expired key gives `401 revoked_api_key` or `401 expired_api_key`. See [Errors](/docs/reference/errors/#status-codes). ## Live keys and test keys | Kind | Starts with | Billed | Serving path | Limits | | --- | --- | --- | --- | --- | | Live | `dex_live_` | Yes, from your prepaid balance | GPU, with the fallback when allowed | 300 requests and 300,000 tokens per minute by default | | Test | `dex_test_` | No | GPU only | 30 requests and 40,000 tokens per minute, 250,000 tokens a day per account | See [Rate limits](/docs/reference/rate-limits/) and [Tokens and billing](/docs/reference/billing/). ## Scopes A key carries one or more scopes. A new key gets all three unless you narrow it in the console. | Scope | Allows | | --- | --- | | `decide` | `POST /v1/decide` | | `usage:read` | `GET /v1/usage` | | `balance:read` | `GET /v1/balance` | `GET /v1/models` and `GET /v1/key` work with any valid key, whatever its scopes. A key without the scope a route needs gets `403 insufficient_scope`. ## Check which key you are using `GET /v1/key` returns the key that made the call: its id, display prefix, mode, scopes, account and creation time. It never returns the secret. Use it to confirm a deployment has the right key. ```bash tab="curl" curl https://api.thinqit.ai/v1/key \ -H "authorization: Bearer $DEX_API_KEY" ``` ```python tab="Python" from thinqit_dex import Client key = Client().get_key() print(key.prefix, key.mode, key.scopes, key.account_id) ``` ```ts tab="TypeScript" import { Client } from "@thinqit/dex"; const key = await new Client().getKey(); console.log(key.prefix, key.mode, key.scopes, key.account_id); ``` ```bash tab="CLI" dex whoami ``` ```json { "object": "key", "id": "key_01M54VQCG06CQ643DZVMXXQKFB", "prefix": "dex_live_Q7mK", "mode": "live", "scopes": ["decide", "usage:read", "balance:read"], "account_id": "acc_01M4ZPXYG0F5KZNWJ47TAN9ZT2", "created": 1792225800 } ``` The values above are illustrative. See [GET /v1/key](/docs/api/getCurrentKey/) in the API reference. ## Key format and checksum A key is a mode prefix followed by 40 base62 characters: 34 random characters (about 202 bits) and a 6-character checksum. The checksum lets secret scanners and your own tools reject a mistyped key without calling the API. - **Alphabet.** `0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz`, with digit values 0 to 61 in that order. - **Shape.** The key matches `^dex_(live|test)_([0-9A-Za-z]{34})([0-9A-Za-z]{6})$`. - **Checksum input.** The UTF-8 bytes of the mode prefix plus the 34 random characters (43 bytes). - **Checksum.** CRC-32 as in IEEE 802.3 and zlib, as an unsigned 32-bit number, written in base62 with the most significant digit first and left-padded with `0` to exactly 6 characters. - **Test vectors.** `dex_live_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA0JnKxX` and `dex_test_0123456789abcdefghijABCDEFGHIJxyzX1wdXRa` are well formed. Changing any one character of either breaks the checksum. A matching checksum proves only that the key was not mistyped. It does not prove that the key exists or is active. Only the API can tell you that, for example with `GET /v1/key`. ```python tab="Python" import re import zlib ALPHABET = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz" KEY = re.compile(r"^(dex_(?:live|test)_)([0-9A-Za-z]{34})([0-9A-Za-z]{6})$") def key_is_well_formed(key: str) -> bool: m = KEY.fullmatch(key) if not m: return False crc = zlib.crc32((m.group(1) + m.group(2)).encode("utf-8")) digits = "" for _ in range(6): crc, r = divmod(crc, 62) digits = ALPHABET[r] + digits return digits == m.group(3) ``` ```ts tab="TypeScript" const ALPHABET = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"; const KEY = /^(dex_(?:live|test)_)([0-9A-Za-z]{34})([0-9A-Za-z]{6})$/; function crc32(bytes: Uint8Array): number { let crc = 0xffffffff; for (const b of bytes) { crc ^= b; for (let k = 0; k < 8; k++) crc = crc & 1 ? (crc >>> 1) ^ 0xedb88320 : crc >>> 1; } return (crc ^ 0xffffffff) >>> 0; } export function keyIsWellFormed(key: string): boolean { const m = KEY.exec(key); if (!m) return false; let n = crc32(new TextEncoder().encode(m[1] + m[2])); let digits = ""; for (let i = 0; i < 6; i++) { digits = ALPHABET[n % 62] + digits; n = Math.floor(n / 62); } return digits === m[3]; } ``` The CLI runs this check before it stores or sends a key: `dex login` refuses a mistyped key with the error code `invalid_key_checksum`. ## Keeping keys safe - **Shown once.** The console shows a key's secret only when you create it. Dex stores only a SHA-256 hash of the key and a display prefix, such as `dex_live_Q7mK`. - **Revocation is fast.** A key you revoke in the console stops working on every gateway replica within about a second. - **Expiry and limits.** A key can have an expiry date and its own limits, at or below the account caps. - **Leaked keys.** We are registering the key format with GitHub secret scanning. Once that is in place, a key found in a public repository is revoked automatically and the account owner gets an email. - **Test keys for development.** Use a test key on development machines and in CI. It is never billed. --- # Errors > The error envelope, every status code with its type and codes, which errors to retry, refunds, and the SDK error classes. Every error is a JSON body with a stable `type` and `code`. The `type` tells you the class of problem and maps one to one to the HTTP status. The `code` tells you the exact reason. Errors before admission never charge your balance, and server errors after admission are refunded. ## The error envelope ```json { "error": { "type": "validation", "code": "state_path_not_found", "message": "Question 'spoed' refers to {{bericht.txt}}, which is not in the state.", "param": "questions.spoed.instructions", "request_id": "req_01M5CJXHG0M9S346Q3D25VT4F5" } } ``` | Field | Always present | Meaning | | --- | --- | --- | | `type` | yes | The class of error. Stable, and one to one with the HTTP status. | | `code` | yes | The machine-readable reason. Stable. New codes may be added within v1, so handle unknown codes by their `type`. | | `message` | yes | English text for people. Do not parse it. | | `param` | no | The input field that caused the error, as a dotted path, when one field did. | | `request_id` | yes | The request id, the same as the `x-request-id` header. Quote it to support. | | `balance_micro_cents` | 402 only | Your balance when the request arrived. | | `required_micro_cents` | 402 only | What the request would have cost. | ## Status codes | HTTP | `type` | Codes | Retry | | --- | --- | --- | --- | | 400 | `invalid_request` | `invalid_json`, `duplicate_key`, `unsupported_media_type`, `invalid_header` | No | | 401 | `authentication` | `missing_api_key`, `invalid_api_key`, `revoked_api_key`, `expired_api_key`, `invalid_session`, `invalid_code`, `code_attempts_exceeded` | No | | 402 | `insufficient_balance` | `insufficient_balance` | After a top-up | | 403 | `permission` | `insufficient_scope`, `account_suspended`, `csrf_failed` | No | | 404 | `not_found` | `model_not_found`, `model_retired`, `route_not_found`, `key_not_found` | No | | 405 | `method_not_allowed` | `method_not_allowed` | No | | 409 | `conflict` | `idempotency_key_reused`, `idempotency_in_progress` | `idempotency_in_progress` only, after `retry-after` | | 413 | `too_large` | `body_too_large`, `state_too_long`, `question_too_long`, `request_too_long` | No | | 422 | `validation` | `unknown_field`, `missing_field`, `invalid_type`, `invalid_value`, `field_not_allowed`, `invalid_question_id`, `too_many_questions`, `too_many_options`, `invalid_levels`, `invalid_min_confidence`, `duplicate_label`, `state_path_not_found`, `state_not_json`, `top_up_limit_exceeded` | No | | 429 | `rate_limited` | `requests_per_minute`, `tokens_per_minute`, `concurrency`, `test_daily_quota`, `email_codes_per_hour` | Yes, after `retry-after` | | 500 | `internal` | `internal_error` | Yes, with the same idempotency key | | 503 | `unavailable` | `no_capacity`, `upstream_unavailable`, `billing_unavailable`, `maintenance` | Yes, after `retry-after` | ### Codes in detail | Code | Status | When | | --- | --- | --- | | `invalid_json` | 400 | The body is empty, not UTF-8 or not JSON. This includes a `\u` escape that is half of a surrogate pair, such as `"\ud800"` without the low half that must follow it. A pair such as `"\ud83d\ude00"`, one emoji, is fine. | | `duplicate_key` | 400 | An object in the body repeats a key, including a label repeated character for character in `options`. `param` is the JSON pointer of the repeated member, for example `/questions/afdeling/options/garantie`. | | `unsupported_media_type` | 400 | A request with a body did not send `content-type: application/json`. | | `invalid_header` | 400 | `idempotency-key` or `x-client-request-id` is empty, too long or not printable ASCII. `param` names the header. | | `missing_api_key` | 401 | No `authorization` header. | | `invalid_api_key` | 401 | The key is not a valid key. | | `revoked_api_key` | 401 | The key was revoked. Create a new one in the console. | | `expired_api_key` | 401 | The key passed its expiry date. | | `invalid_session` | 401 | Console only: the session cookie is missing or no longer valid. | | `invalid_code` | 401 | Console sign-in only: the code is wrong or has expired. Check the newest email, or ask for a new code. | | `code_attempts_exceeded` | 401 | Console sign-in only: the code had 5 wrong tries and no longer works, even with the right digits. Ask for a new code. | | `insufficient_balance` | 402 | The balance does not cover this request. Nothing was charged. | | `insufficient_scope` | 403 | The key lacks the scope for this route, such as `usage:read`. | | `account_suspended` | 403 | The account is suspended. | | `csrf_failed` | 403 | Console only: the `x-dex-csrf` header is missing or wrong. | | `model_not_found` | 404 | No model has this name. See `GET /v1/models`. | | `model_retired` | 404 | This exact version has been retired. | | `route_not_found` | 404 | No such route. | | `key_not_found` | 404 | No key with this id in your account. | | `method_not_allowed` | 405 | The path exists, but not with this method, for example `GET /v1/decide`. The `allow` header lists the methods it has. | | `idempotency_key_reused` | 409 | This idempotency key was used in the last 24 hours with a different request body. | | `idempotency_in_progress` | 409 | The first request with this idempotency key is still running. | | `body_too_large` | 413 | The body is over 256 KiB. | | `state_too_long` | 413 | The rendered state is over 16,384 tokens. | | `question_too_long` | 413 | A rendered question is over 4,096 tokens. | | `request_too_long` | 413 | The request is over 16,384 billable tokens in total, or its questions together are over 4,096 (then `param` is `questions`). | | `unknown_field` | 422 | A field the API does not define, for example the typo `questons`. | | `field_not_allowed` | 422 | A field of another question type: `options` outside a pick, `levels` outside a rate. | | `missing_field` | 422 | A required field is absent, for example `questions`, a question's `type` or `instructions`, or the `start` query parameter of `getUsage`. | | `invalid_type` | 422 | A value has the wrong JSON type: a number for `instructions`, an array for `questions`, a number for the state. | | `invalid_value` | 422 | The type is right but the value is not allowed: an empty or over-long string (the state, `instructions`, `criteria`, a label, a description), a state, `instructions`, `criteria` or description of only whitespace, an empty `questions` or `options` object, an empty state object or array, a string outside its allowed values or pattern (`model`, `fallback`, an unknown question `type`, a label or a rate level with spaces at either end), a JSON state nested deeper than 32 levels or with an empty or over-long key. For a level, `param` names the level, such as `questions.spoed.levels.2`. On `getUsage`: a malformed date, `granularity` or `key_id`, an `end` not after `start`, or a range longer than the granularity allows. | | `invalid_question_id` | 422 | A question id does not match `^[A-Za-z][A-Za-z0-9_-]{0,63}$`. | | `too_many_questions` | 422 | More than 32 questions. | | `too_many_options` | 422 | A pick has more than 255 options. | | `invalid_levels` | 422 | `levels` is not 2 to 10 unique strings of 1 to 200 characters. Levels must differ by more than case or Unicode normalization: `Low` and `low` count as the same level. | | `invalid_min_confidence` | 422 | `min_confidence` is not a number from 0 to 1. | | `duplicate_label` | 422 | Two pick labels differ only in case or Unicode normalization, such as `Billing` and `billing`, or a precomposed and a combining `é`. A label repeated character for character is `duplicate_key` (400). | | `state_path_not_found` | 422 | A `{{path}}` reference names a field the state does not have. | | `state_not_json` | 422 | A `{{path}}` reference appears while the state is a string. | | `top_up_limit_exceeded` | 422 | Console only: a top-up would take a new account past EUR 500 in its first 30 days. | | `requests_per_minute` | 429 | The requests-per-minute limit is used up. | | `tokens_per_minute` | 429 | The billable-tokens-per-minute limit is used up. | | `concurrency` | 429 | Too many requests are in flight at once. | | `test_daily_quota` | 429 | The account used its free test tokens for today. `retry-after` is the time until they reset at 00:00 UTC. The SDKs do not retry it. | | `email_codes_per_hour` | 429 | Console sign-in only: this address was sent 5 codes in the last hour, or too many codes were asked for from one network. `retry-after` is the real wait, up to an hour. | | `internal_error` | 500 | Something went wrong on our side. Any charge was refunded. | | `no_capacity` | 503 | No GPU capacity, and the fallback is not allowed or not available for this request. | | `upstream_unavailable` | 503 | The GPU path and the fallback both failed. | | `billing_unavailable` | 503 | The balance store could not be reached. | | `maintenance` | 503 | The service is in maintenance. | ## Which error comes first Dex checks a request in this order and stops at the first failure: 1. Authentication (401), then scope (403). 2. Body size (413 `body_too_large`), content type (400 `unsupported_media_type`), then JSON parsing (400 `invalid_json`). 3. Duplicate keys (400 `duplicate_key`), then the schema (422, with a code per rule). 4. The model name (404). 5. Field references, and label and level uniqueness ignoring case (422). 6. Rendering and token limits (413). 7. Idempotency (409). 8. Rate limits and quotas (429). 9. Balance (402). 10. Dispatch to a serving path (503 or 500). Steps 1 to 9 never charge your balance. When a body breaks several schema rules at once, the most specific code wins, in this order: `unknown_field`, `field_not_allowed`, `missing_field`, `invalid_question_id`, `too_many_questions`, `too_many_options`, `invalid_levels`, `invalid_min_confidence`, `invalid_value`, `invalid_type`. For a 422, `param` is a dotted path into the body, such as `questions.spoed.instructions`, or the name of a query parameter. For `duplicate_key`, `param` is a JSON pointer. New codes can appear within v1 under an existing `type`. Act on the `type`, and treat an unknown `code` like the other codes of its type. ## Retrying - **429, 503 and 409 `idempotency_in_progress`** always carry a `retry-after` header in whole seconds, from 1 to 60. Wait that long, then retry. Two codes wait longer: `test_daily_quota` gives the time until 00:00 UTC, and the console's sign-in code limit gives the real wait, up to 3,600 seconds. - **500** is safe to retry with the same idempotency key. - **402** succeeds after you top up. - **Every other 4xx** will fail again. Fix the request instead. - **Send an `idempotency-key`** on `POST /v1/decide` so a retry can never charge twice. See [Idempotency](/docs/reference/idempotency/). - **Back off** when there is no `retry-after`, for example after a network error. The SDKs retry 429 (but not `test_daily_quota`), 500, 502, 503, 504, 409 `idempotency_in_progress`, timeouts and network errors, with full-jitter exponential backoff from a 250 ms base, at most 3 retries. - **Watch for duplicate keys** if you build JSON by hand or merge objects. A repeated key is a 400 `duplicate_key`, never silently the last value. ## Refunds - Errors before admission (all 4xx) never charge. A 402 means nothing was charged. - A request that fails with 500 or 503 after the charge is refunded in full before the response is sent. The tokens it took from your tokens-per-minute limit and, for a test key, from the daily test quota are given back too. - A replay of a completed request with the same idempotency key is never charged. It returns the original response, including the original `usage`. ## SDK error classes Until the SDKs are on PyPI and npm, install them from [Downloads](/docs/reference/sdks/#downloads). Each SDK raises one class per error `type`. The class comes from `error.type` in the body, or from the HTTP status when the body cannot be read. | `type` | HTTP | Class | | --- | --- | --- | | `invalid_request` | 400 | `InvalidRequestError` | | `authentication` | 401 | `AuthenticationError` | | `insufficient_balance` | 402 | `InsufficientBalanceError`, with `balance_micro_cents` and `required_micro_cents` | | `permission` | 403 | `PermissionDeniedError` | | `not_found` | 404 | `NotFoundError` | | `conflict` | 409 | `ConflictError` | | `too_large` | 413 | `TooLargeError` | | `validation` | 422 | `RequestValidationError` | | `rate_limited` | 429 | `RateLimitError` | | `internal` | 500 | `InternalServerError` | | `unavailable` | 503 | `UnavailableError` | All of these extend `APIStatusError`, which carries `status`, `type`, `code`, `message`, `param`, `request_id`, `retry_after`, `idempotency_key` and the response `headers`. An unknown `type` raises a plain `APIStatusError`, and so does a 405 `method_not_allowed`, which no SDK call can cause. Errors without an HTTP answer have their own classes: `APIConnectionError` (with `APITimeoutError` under it), `ResponseParseError` for a body that is not valid JSON, and `ConfigurationError`, for example when no API key is set. The TypeScript SDK adds `AbortedError` when your `AbortSignal` stops a call. Every class extends `DexError`. ```python tab="Python" from thinqit_dex import RequestValidationError, InsufficientBalanceError try: decision = client.decide(state=state, questions=questions) except RequestValidationError as err: print(err.code, err.param, err.request_id) except InsufficientBalanceError as err: print(err.required_micro_cents - err.balance_micro_cents, "micro-cents short") ``` ```ts tab="TypeScript" import { RequestValidationError, InsufficientBalanceError } from "@thinqit/dex"; try { const decision = await client.decide({ state, questions }); } catch (err) { if (err instanceof RequestValidationError) { console.log(err.code, err.param); } else if (err instanceof InsufficientBalanceError) { console.log("Top up in the console:", err.message); } else { throw err; } } ``` ## Getting help Every response and every error has a request id: `req_` followed by 26 characters. Quote it when you contact support. Support asks for the request id, never for the content of your request. --- # Headers > The request and response headers of the API, including request ids, idempotency and the rate limit headers. This page lists every header the API reads or sends. Header names are case-insensitive. The examples use lowercase. ## Request headers | Header | Required | Rule | | --- | --- | --- | | `authorization: Bearer ` | Yes, on every public route: `/v1/decide`, `/v1/models`, `/v1/key`, `/v1/usage` and `/v1/balance` | Your API key, starting with `dex_live_` or `dex_test_`. Missing or bad keys give 401. | | `content-type: application/json` | Yes, on every request with a body | The body must be JSON in UTF-8. Anything else gives `400 unsupported_media_type`. | | `idempotency-key` | No | On `POST /v1/decide` and the console's top-up route. 1 to 255 printable ASCII characters, without spaces. A malformed key gives `400 invalid_header`. See [Idempotency](/docs/reference/idempotency/). | | `x-client-request-id` | No | Your own correlation id, on every public operation. 1 to 128 printable ASCII characters, without spaces. It is stored with the request metadata and echoed in the response. A malformed value gives `400 invalid_header` on every route. | | `user-agent` | No | The SDKs send their name and version. Dex keeps the SDK name and version with the request metadata. | | `x-dex-csrf` | Console only | The token the console receives at sign-in. Required on every console request that changes something. | ## Response headers | Header | When | Rule | | --- | --- | --- | | `x-request-id` | Always | The request id: `req_` followed by a 26-character ULID. The same value is in the body's `id` and in `error.request_id`. | | `x-client-request-id` | When you sent a valid one | Your correlation id, echoed on every response, errors included. | | `idempotent-replayed: true` | On a replay | The idempotency key matched an earlier completed request. The body is the original response, and nothing is charged again. The `x-request-id` header names the replay itself. See [Idempotency](/docs/reference/idempotency/). | | `ratelimit-policy` | Every authenticated API response | The limits that apply to this key. See [Rate limit headers](#rate-limit-headers). | | `ratelimit` | Every authenticated API response | What is left of each limit, and when it resets. | | `retry-after` | On 429, 503 and 409 `idempotency_in_progress` | Whole seconds to wait before retrying, from 1 to 60. | ## Rate limit headers The rate limit headers use the structured-field form of the IETF httpapi rate limit draft. Each lists one entry per limit, separated by commas: ```text ratelimit-policy: "rpm";q=300;w=60, "tpm";q=300000;w=60 ratelimit: "rpm";r=287;t=12, "tpm";r=291400;t=12 ``` | Name or parameter | Meaning | | --- | --- | | `"rpm"` | Requests per minute. | | `"tpm"` | Billable tokens per minute. | | `q` | The quota: how many requests or tokens the window allows. | | `w` | The window, in seconds. | | `r` | What remains of the quota. | | `t` | Seconds until the quota resets. | Read the example as: this key may send 300 requests and 300,000 billable tokens per 60 seconds. It has 287 requests and 291,400 tokens left, and both reset in 12 seconds. Use `ratelimit` to slow down before you hit a limit. When you do hit one, the 429 response names the exhausted limit in its error `code` and carries `retry-after`. See [Rate limits](/docs/reference/rate-limits/). ## Request ids The gateway gives every request an id, including requests it rejects. The id is `req_` plus a ULID, so ids sort by time. You find it in the `x-request-id` header, in the body's `id` and in `error.request_id`. Log it next to your own records. Support asks for the request id, never for the content of a request. If you send `x-client-request-id`, it is stored next to the request id in the request metadata and echoed back on the response, whichever public route you call and whether the call succeeds or fails. The SDKs take it on every method (`client_request_id` in Python, `clientRequestId` in TypeScript). --- # Rate limits > Request, token and concurrency limits per key and per account, how they are counted, and what to do on a 429. Each API key has three limits: requests per minute, billable tokens per minute and requests in flight at once. Test keys also share a daily token quota per account. When you hit a limit, you get a 429 that names the limit and tells you how long to wait. ## Limits | Limit | Live key default | Test key | Account cap (live) | | --- | --- | --- | --- | | Requests per minute | 300 | 30 | 600 | | Billable tokens per minute | 300,000 | 40,000 | 600,000 | | Concurrent requests | 16 | 4 | 32 | | Tokens per day | No limit | 250,000 per account | No limit | - **Per key.** A new live key starts at the defaults. In the console you can set a key's limits lower, at or below the account caps: 1 to 600 requests per minute, 1,000 to 600,000 tokens per minute, and 1 to 32 concurrent requests. - **Account cap.** The ceiling for your account's live limits. To go above it, ask for a raise (see [Raising limits](#raising-limits)). - **Test quota.** The daily test quota counts the billable tokens of every test key and every console playground call of the account together, per UTC day. ## How limits are counted - **Requests per minute** counts every call the key makes to a public operation: `decide`, `listModels`, `getCurrentKey`, `getUsage` and `getBalance`, each one request. Idempotent replays count too. - **Tokens per minute** counts billable tokens, the same `input_tokens` you are billed for. Only `decide` uses it. Dex knows this count before it runs the request, so the limit is charged with the exact number. - **Concurrent requests** counts `decide` requests that have been admitted and have not yet answered. The per-minute limits work like a bucket that refills at a steady rate, not like a counter that resets on the minute. You can send a short burst up to the quota, and capacity comes back gradually after it. All three limits are checked together in one step for every request, so a request is either admitted under all of them or rejected. **The largest request always fits a default limit.** A request bills at most 16,384 tokens, and the smallest default tokens-per-minute limit is the test key's 40,000. So a key with a full bucket admits any valid request. If you lower a key's limit below 16,384 (the minimum is 1,000), a request larger than that limit is admitted only when the bucket is full, and it empties the bucket. Current values are in the `ratelimit-policy` and `ratelimit` headers of every authenticated response. See [Headers](/docs/reference/headers/#rate-limit-headers). ## When you hit a limit You get HTTP 429 with type `rate_limited`, a `code` that names the limit, and a `retry-after` header in whole seconds: | Code | Limit | What to do | | --- | --- | --- | | `requests_per_minute` | Requests per minute | Wait `retry-after` seconds. Spread requests out, or ask more questions per request. | | `tokens_per_minute` | Billable tokens per minute | Wait `retry-after` seconds. Send smaller states, or spread requests out. | | `concurrency` | Concurrent requests | Wait for requests in flight to finish. Lower the concurrency of your workers. | | `test_daily_quota` | Daily test quota | Wait for the next UTC day (`retry-after` is the time until 00:00 UTC), or use a live key. | ```json { "error": { "type": "rate_limited", "code": "tokens_per_minute", "message": "This key used its 300000 tokens for this minute. Retry in 7 seconds.", "request_id": "req_01M5CJXHG0M9S346Q3D25VT4F5" } } ``` A 429 is never charged. The SDKs retry it for you after `retry-after`, up to 3 times, except `test_daily_quota`, which they raise at once. A request that fails with 500 or 503 gives back the tokens it took from your tokens-per-minute limit and from the daily test quota. Requests per minute stay counted. Because each question in a request shares one state, one request with five questions uses fewer requests and fewer tokens than five requests with one question each. ## Degraded mode Limits are counted in a shared store. If that store is unreachable, each gateway replica enforces its share of your limit (the limit divided by the number of live replicas) on its own. Your limits still apply, counted per replica, so they hold approximately rather than exactly while the store is down. ## Fair sharing The queue in front of the GPU is weighted-fair per account. One account at its cap cannot starve other accounts. Live keys go ahead of test keys and playground calls. ## Raising limits Ask for higher limits in the console. We raise account caps in steps, and each step is no bigger than the spare capacity the GPU pool showed over the last 7 days. --- # Idempotency > Retry a request safely with an idempotency key, so a network failure never makes you pay twice. When a request times out or the connection drops, you cannot tell whether it ran. Send an `idempotency-key` header, and you can retry with the same key without being charged twice. A replay of a completed request is never charged. ## How to use it Send the header on `POST /v1/decide`. The console's top-up route accepts it too. ```bash curl https://api.thinqit.ai/v1/decide \ -H "authorization: Bearer $DEX_API_KEY" \ -H "content-type: application/json" \ -H "idempotency-key: 7f3c1e9a-2b4d-4c8e-9a61-0d5f2e8b7c43" \ -d @request.json ``` - A key is 1 to 255 printable ASCII characters, without spaces. A malformed key gives `400 invalid_header`. - A UUID is a good choice. - Use one key per logical request, and send the same key on every retry of that request. - Use a new key for a new request, even when its body is the same as an earlier one. ## What happens to the key - A key is scoped to the API key that sent it (to the account, for console routes). Two of your API keys can use the same idempotency key without clashing. - Dex keeps each idempotency key for 24 hours. - It stores a small record: your key id, a hash of the idempotency key, a fingerprint of the request (a SHA-256 hash of its canonical form), the status and timestamps. A completed request adds its original `id` and `created`, the model version, `served_by`, the token counts and the charge fields. It stores no request content and no answers. ## Replays When a request with the same key and the same body has already completed, you get **the original response**, and nothing is charged again: - The response carries `idempotent-replayed: true`. - `id`, `created` and `usage` are the original ones, including the original charge fields. They show what the first call cost; nothing is debited now. - The `x-request-id` header names the replay itself, so support can find both calls. - On a GPU version, the answers are computed again on the original exact version and are the same, byte for byte, as the first time. See [Determinism](/docs/concepts/determinism/). A replay never goes to the fallback: if no GPU can serve that version, you get `503 no_capacity` with `retry-after`, and nothing is charged. - **One exception: an original served by the fallback.** Its answers cannot be reproduced and are not stored. The replay runs the request again on the normal path, without charging. Its answers, `model` and `served_by` can differ from the original's. `id`, `created` and `usage` are still the original's. In short: idempotency makes sure you are charged at most once, and a replay looks exactly like the call it retries. ## Conflicts | Situation | Response | What to do | | --- | --- | --- | | Same key, different body | `409 idempotency_key_reused` | Do not retry. You reused a key for a different request. Use a new key. | | Same key, first request still running | `409 idempotency_in_progress` with `retry-after: 1` | Wait, then retry with the same key. | ## After a failure - **5xx.** A request that ended in a 5xx is recorded as failed, and its charge was refunded. A retry with the same key runs again normally, and it is charged once if it succeeds. - **429 and 402.** A request stopped by a rate limit or by your balance keeps no record. A retry with the same key, after `retry-after` or after a top-up, runs as a new request. - **Other 4xx.** A 400, 401, 403, 404, 413 or 422 is rejected before the idempotency step and never creates a record. Fix the request, and use a new key for the fixed request, because its body is different. A 4xx is never charged. Idempotency records live in a fast store without persistence in v1. A failover of that store forgets them. While it is unreachable, a `decide` request that carries an idempotency key gets `503 billing_unavailable`, so a retry can never be charged twice. ## What counts as the same body Dex compares requests by their canonical form: the parsed JSON body with object keys in their original order, no insignificant whitespace, strings in Unicode NFC form and numbers in their shortest form. - Reformatting the JSON, such as adding spaces or line breaks, does not change the fingerprint. - Changing the order of keys, including the options of a pick, does change it. ## SDKs Until the SDKs are on PyPI and npm, install them from [Downloads](/docs/reference/sdks/#downloads). The SDKs send an idempotency key on every `decide` call without being asked. They use your key if you pass one, or a new UUID v4 otherwise, and reuse it on every retry of that call. Read-only calls send none. Pass your own key when a retry can happen in a different process, for example after a worker restart, so the retry still matches: ```python tab="Python" decision = client.decide(state=state, questions=questions, idempotency_key=job.id) ``` ```ts tab="TypeScript" const decision = await client.decide({ state, questions }, { idempotencyKey: job.id }); ``` --- # Limits > Every size limit of a decide request, the error each one returns, and the model limits from the models endpoint. This page lists every size limit of a `decide` request and the error you get when you pass it. Rate limits are on their own page: [Rate limits](/docs/reference/rate-limits/). ## Request size limits | Limit | Value | Error | | --- | --- | --- | | Request body | 256 KiB | 413 `body_too_large` | | State tokens, after rendering | 16,384 | 413 `state_too_long` | | Tokens per rendered question | 4,096 | 413 `question_too_long` | | Billable tokens per request | 16,384 | 413 `request_too_long` | | Billable tokens of all questions together | 4,096 | 413 `request_too_long`, with `param` `questions` | | Questions per request | 32 (at least 1) | 422 `too_many_questions` | | Options per pick | 255 (at least 1) | 422 `too_many_options` | | Levels per rate | 2 to 10 | 422 `invalid_levels` | Token limits are counted after rendering and tokenizing, with the tokenizer of the requested model family. They match the billed counts exactly: `state_tokens`, the tokens of each question, and `input_tokens` for the whole request. On 26 September 2026 the request limits went down from 32,768 billable tokens (with no limit on the questions together) to the values above, so that every allowed request is answered on the GPU in time. The slowest request they allow, a 12,288-token state with one 4,096-token question, takes about 3 seconds on the GPU. Questions cost more per token than the state, because the model reads all of them in one pass that attends to the whole state. ## State | Limit | Value | | --- | --- | | A string state | 1 to 65,536 characters, not only whitespace | | A JSON object state | At least 1 field | | JSON nesting | At most 32 levels deep | | JSON keys | 1 to 256 characters each | | A JSON array state | At least 1 item | A state that breaks one of these gives 422 `invalid_value`; a state of the wrong JSON type, such as a number, gives 422 `invalid_type`. A key repeated inside the same JSON object gives 400 `duplicate_key`. See [State](/docs/concepts/state/). ## Questions | Field | Limit | Error | | --- | --- | --- | | Question id | Matches `^[A-Za-z][A-Za-z0-9_-]{0,63}$` | 422 `invalid_question_id` | | `instructions` | 1 to 4,000 characters, not only whitespace | 422 `invalid_value` | | `criteria` as a string | 1 to 4,000 characters, not only whitespace | 422 `invalid_value` | | `criteria` as a list | 1 to 20 strings, each 1 to 500 characters and not only whitespace | 422 `invalid_value` | | Pick option label | 1 to 100 characters, no leading or trailing whitespace | 422 `invalid_value` | | Pick option labels | Unique: a label repeated character for character gives 400 `duplicate_key`; labels equal only after Unicode NFC normalization or ignoring case (`Billing` and `billing`) give 422 `duplicate_label` | 400 `duplicate_key`, 422 `duplicate_label` | | Pick option description | `null`, or 1 to 1,000 characters, not only whitespace | 422 `invalid_value` | | Rate level | 1 to 200 characters, no leading or trailing whitespace | 422 `invalid_value` | | Rate levels | 2 to 10 strings, unique after Unicode NFC normalization and ignoring case | 422 `invalid_levels` | | `min_confidence` | 0 to 1 | 422 `invalid_min_confidence` | Where the table shows only the type `validation`, the error `code` names the rule that failed, and `param` names the field. ## Headers | Header | Limit | Error | | --- | --- | --- | | `idempotency-key` | 1 to 255 printable ASCII characters, without spaces | 400 `invalid_header` | | `x-client-request-id` | 1 to 128 printable ASCII characters, without spaces | 400 `invalid_header` | ## Model limits Each model version reports its limits in `GET /v1/models`. The values below are the ones in the contract. Read them from the endpoint rather than hard-coding them: limits may change within v1, as they did on 26 September 2026. The limit on all questions together (4,096 tokens) is not in the endpoint yet. | Field | Value | Meaning | | --- | --- | --- | | `max_state_tokens` | 16384 | State tokens after rendering. | | `max_question_tokens` | 4096 | Tokens per rendered question. | | `max_total_tokens` | 16384 | Billable tokens per request. | | `max_questions` | 32 | Questions per request. | | `max_options` | 255 | Options per pick. | | `max_levels` | 10 | Levels per rate. | | `exact_option_probabilities` | 255 on GPU versions, 20 on fallback versions | How many options of a pick get exact probabilities. See [Fallback and served_by](/docs/concepts/fallback/#what-differs-on-the-fallback). | ## Other limits | Limit | Value | | --- | --- | | `GET /v1/usage` range | Up to 93 days by day, up to 7 days by hour | | Top-up amount | EUR 10 to EUR 2,500 per top-up | | Top-ups in an account's first 30 days | EUR 500 in total (`422 top_up_limit_exceeded`) | | Opt-in content logging | 1 to 30 days of retention | | API key name | 1 to 64 characters | --- # Tokens and billing > Output is free and you pay for input tokens only, EUR 0.05 per million. What counts, how a charge is computed and the prepaid balance. Output is free: you pay for input tokens only. Every response tells you exactly how many tokens you were billed for and what the call cost, and you know the charge before the request runs. The price is EUR 0.05 per million input tokens (5 micro-cents per token), flat at every volume, excluding VAT. Billing starts when the API opens to the public. ## What you pay for Every response has a `usage` object: | Field | Meaning | | --- | --- | | `input_tokens` | The billed tokens: `state_tokens` plus `question_tokens`. | | `state_tokens` | The tokens of the rendered state. | | `question_tokens` | The tokens of all rendered questions: instructions, criteria, and options or levels with their answer codes. | | `charge_micro_cents` | What was debited from your balance. | | `unit_price_micro_cents` | The price per token for this request: 5 at launch, or 0 for a test key. | | `tier` | `t1`, `t2`, `t3` or `test`. At launch t1, t2 and t3 have the same price. | Three rules follow: - **Output is free.** The answers are never counted. - **Scaffolding is free.** The fixed prompt text Dex adds around your content (its prefix, delimiters and answer cue) is not billed. - **The state is billed once.** However many questions share it. See [State](/docs/concepts/state/#sent-once-billed-once). ## The tokenizer Tokens are counted with the pinned tokenizer of the model family: the tokenizer of the base model that `dex-1` resolves to. It stays the same for every version in major version 1, because a new tokenizer means a new major version. Requests served by the fallback are counted with the same tokenizer, so a request costs the same on either path. ## The charge ```text charge_micro_cents = input_tokens * unit_price_micro_cents ``` - One euro is 100,000,000 micro-cents. - Prices are whole micro-cents per token, so every charge is an exact whole number. Nothing is rounded. - Because output is free, the exact charge is known before the request runs. ## Price and tiers Prices are per million input tokens, excluding VAT. The API keeps volume tiers, counted per account on billable tokens since the start of the UTC calendar month, so that volume discounts can be added later without a contract change. The price is flat: the same at every volume. | Tier | Billable input tokens this month | EUR per million input tokens | Micro-cents per token | | --- | --- | --- | --- | | t1 | first 1 billion | 0.05 | 5 | | t2 | 1 billion to 10 billion | 0.05 | 5 | | t3 | above 10 billion | 0.05 | 5 | The launch price is flat: EUR 0.05 per million input tokens at every volume. The API still reports a `tier` (t1, t2 or t3) on every response; at launch every tier has the same price. Prices exclude VAT. Output tokens are free. - A request is priced at the tier in force before it. - `GET /v1/balance` shows your month-to-date tokens, your tier, its unit price and `next_tier_at_tokens`, the count at which the next tier starts. - The [pricing page](/pricing/) shows the same price. ## Worked examples | Call | Billable tokens | Charge | | --- | --- | --- | | One check on a short message | 150 | EUR 0.0000075 | | The Dutch support example in the docs | 273 | EUR 0.00001365 | | A typical 450-token call | 450 | EUR 0.0000225 | | A 1,000-token state with 8 questions of 60 tokens | 1,480 | EUR 0.000074 | One million calls of 450 tokens cost EUR 22.50. ## Test keys Test keys (`dex_test_`) are counted the same way and charged 0, with `tier` `test`. Each account gets 250,000 free test tokens per UTC day, shared by all its test keys and by console playground calls. Test calls always run on the GPU path. See [Rate limits](/docs/reference/rate-limits/). ## Prepaid balance Dex is prepaid. You top up a balance in euros, and each request is debited from it. - **Stored exactly.** The balance is an integer number of micro-cents. There are no credits or rounding. - **Top-ups** of EUR 10 to EUR 2,500 go through Stripe Checkout in the console. - **Payment methods:** cards, iDEAL, Bancontact and SEPA Direct Debit. - **New accounts** can top up at most EUR 500 in total in their first 30 days. - **Credited when paid.** The net amount, excluding VAT, is credited when Stripe reports the payment as settled. SEPA Direct Debit settles later than cards, so the balance arrives later too. - **VAT.** Prices exclude VAT. Stripe Tax applies it at checkout: the EU reverse charge for businesses with a validated VAT id, and 21% Dutch VAT for customers in the Netherlands. - **No expiry.** Balances do not expire. When you close your account, you can ask for the unused balance to be refunded. ## When you are charged 1. Dex validates the request and counts its tokens. A request that fails validation (any 4xx) is never charged. 2. At admission, it debits the exact charge in one atomic step. If your balance is too low, you get `402 insufficient_balance` with `balance_micro_cents` and `required_micro_cents`, and nothing is charged. 3. If the request then fails with a 500 or 503, the charge is refunded in full before the response is sent. A replay of a completed request with the same [idempotency key](/docs/reference/idempotency/) is never charged. ## Checking usage and balance - `GET /v1/balance` returns your balance in micro-cents and as a decimal euro string, your tier and your month-to-date tokens. See the [API reference for getBalance](/docs/api/getBalance/). - `GET /v1/usage` returns usage per key by day (up to 93 days) or by hour (up to 7 days), with requests, tokens, charges and the split between GPU and fallback. See the [API reference for getUsage](/docs/api/getUsage/). - The console shows the same data as charts. --- # Data handling and residency > Where requests are processed, what is stored and for how long, the fallback exception, roles under the GDPR and security basics. Dex processes every request in the EU, and by default it stores no request content at all. It keeps request metadata for 30 days and billing records for 7 years. Customer content is never used to train, tune or calibrate a model. ## Where data is processed | Part | Where | | --- | --- | | Gateway, database, cache, secrets and logs | Microsoft Azure, West Europe region (the Netherlands) | | Dex inference nodes | Dedicated hardware operated by thinQit in the Netherlands | | Fallback | Azure OpenAI in the EU data zone (Data Zone Standard), with its account in West Europe, or Sweden Central if West Europe lacks the model | | Payments | Stripe, through Stripe Payments Europe in Ireland. Stripe processes payment data only, never request content. | The full list of third parties is on the [subprocessors page](/legal/subprocessors/). ## No content stored by default Your state, questions, labels and answers exist only in memory while the request runs: in the gateway, on the GPU worker, in the relay between gateway replicas (an in-memory store with persistence turned off) and, when used, in the fallback call. - They are never written to disk, logs, traces, metrics or error reports. - Logs and traces are built from an allow-list of fields, so a field that carries content cannot be logged by accident. - The GPU worker holds content in GPU and host memory only while the job runs. - Responses are not cached across requests. - The console playground does not store content either. Support asks for a request id, never for request content. ## What we keep, and for how long **Request metadata, 30 days.** For each request: - the request id, timestamps, account id and key id; - the model and `served_by`; - token counts and the charge; - the latency breakdown, status and error code; - the number of questions of each type; - the SDK name and version from `user-agent`; - your `x-client-request-id`, if you sent one; - the client IP address, truncated to /24 for IPv4 or /48 for IPv6. **Billing records, 7 years.** Usage totals per minute, ledger entries, top-ups and invoices, as Dutch tax law requires. **Idempotency records, 24 hours.** A fingerprint of the request and its outcome, never its content. See [Idempotency](/docs/reference/idempotency/#what-happens-to-the-key). ## Opt-in content logging Content logging is off unless the account owner turns it on. It helps when you debug an integration. - The owner turns it on in the console, separately for live and test keys, with a retention of 1 to 30 days (7 by default). - Logged requests and responses are encrypted with a data key for your account, which is itself protected by a key in Azure Key Vault, and stored in Azure Blob storage in the EU. - They are deleted automatically when the retention period ends. - The console shows them for debugging. - Turning logging off deletes the existing logs within 1 hour. ## No training on customer data Customer content is never used to train, tune or calibrate any model. Calibration uses our own decision sets. See [Calibration](/docs/concepts/calibration/#data-and-reference-labels). ## Azure OpenAI abuse monitoring This exception applies only to requests served by the fallback. By default, Azure OpenAI abuse monitoring may store prompts that its classifiers flag, for up to 30 days, inside the EU data zone. Before launch we apply to Microsoft for modified abuse monitoring. Until Microsoft approves it, the exception applies, and the data processing agreement states it too. To avoid it, keep your requests on the GPU path: send `"fallback": "never"`, or pin an exact model version. See [Fallback and served_by](/docs/concepts/fallback/#keeping-a-request-on-the-gpu). ## Roles - **Request content:** you are the controller, and Thinqit B.V. is the processor, under the [data processing agreement](/legal/dpa/). - **Account and billing data:** Thinqit B.V. is the controller. See the [privacy policy](/legal/privacy/). The legal documents are drafts until counsel review is complete. ## Security - **Transport.** HTTPS only, with TLS 1.2 or newer. - **API keys.** A key is `dex_live_` or `dex_test_` followed by 40 characters: 34 random characters and a 6-character checksum, so secret scanners can check the format offline. We store only a SHA-256 hash of the key and a short display prefix, and show the full key once, when you create it. - **Scopes and expiry.** Keys have scopes (`decide`, `usage:read`, `balance:read`), an optional expiry date and optional limits at or below your account's caps. - **Revocation.** Revoking a key takes effect across every gateway replica within 1 second. - **Leaked keys.** Before launch we register the key format with GitHub secret scanning, so a key pushed to a public repository is revoked automatically and its owner is emailed. - **Prompt injection.** Instructions written inside the state can still steer answers. Add the guardrail question from [Known limits](/docs/concepts/known-limits/#instructions-inside-the-state-can-steer-answers) and route on it when the state holds text from others. - **Console.** Sign-in is by a one-time email code. The session cookie is HttpOnly, Secure and SameSite=Lax, and every console action that changes something needs a CSRF token. - **Secrets and network.** Our own secrets live in Azure Key Vault, and services reach the database with managed identities, so there is no database password. The database and cache accept private network traffic only, and Dex's inference nodes accept no inbound connections. --- # SDKs and CLI > The Python and TypeScript SDKs, the command line tool and the VS Code extension, with downloads, configuration, retries, idempotency and errors. Dex has two SDKs, one for Python and one for TypeScript, a command line tool and a VS Code extension. They are thin clients over the same API contract: requests look like the JSON you would send with curl, and answers come back as typed objects. The packages are coming soon to PyPI, npm and the VS Code Marketplace. Until then, install the interim builds from [Downloads](#downloads): they are built from the same code and work the same way. **Licence.** The Python SDK, the TypeScript SDK, the CLI and the VS Code extension are open source under the Apache-2.0 licence. You can use them in commercial products, change them and ship them. ## Install Once the packages are published (coming soon), these are the commands. Until then, use [Downloads](#downloads). Python 3.10 or newer: ```bash pip install thinqit-dex ``` TypeScript and JavaScript, for Node.js 18 or newer, Deno, Bun and edge runtimes. The package has no runtime dependencies and ships both ES modules and CommonJS: ```bash npm i @thinqit/dex ``` The command line tool: ```bash npm i -g @thinqit/dex-cli ``` ## Downloads The packages are not on PyPI, npm or the VS Code Marketplace yet. Until they are, install these interim builds from this site. They are built from the same commit as the site you are reading, so they match these docs. | Client | File | | --- | --- | | Python SDK | [`thinqit_dex-0.1.1-py3-none-any.whl`](/downloads/thinqit_dex-0.1.1-py3-none-any.whl) (wheel) and [`thinqit_dex-0.1.1.tar.gz`](/downloads/thinqit_dex-0.1.1.tar.gz) (source) | | TypeScript SDK | [`thinqit-dex-0.1.1.tgz`](/downloads/thinqit-dex-0.1.1.tgz) | | CLI | [`thinqit-dex-cli-0.1.1.tgz`](/downloads/thinqit-dex-cli-0.1.1.tgz) | | VS Code extension | [`thinqit-dex-vscode-0.1.1.vsix`](/downloads/thinqit-dex-vscode-0.1.1.vsix) | The file names carry the version. When a new version is built, this page links the new files. The Python SDK, for Python 3.10 or newer: ```bash pip install https://thinqit.ai/downloads/thinqit_dex-0.1.1-py3-none-any.whl ``` The TypeScript SDK, in your project folder: ```bash npm i https://thinqit.ai/downloads/thinqit-dex-0.1.1.tgz ``` The CLI, for Node.js 18 or newer. It gives you the `dex` command: ```bash npm i -g https://thinqit.ai/downloads/thinqit-dex-cli-0.1.1.tgz dex --version ``` The VS Code extension. Download the file, then install it: ```bash curl -fLO https://thinqit.ai/downloads/thinqit-dex-vscode-0.1.1.vsix code --install-extension thinqit-dex-vscode-0.1.1.vsix ``` You can also install the file from inside VS Code. Open the Extensions view, open the **...** menu at the top, choose **Install from VSIX...** and pick the file. ### Check a download Every file has a `.sha256` file next to it, and [SHA256SUMS.txt](/downloads/SHA256SUMS.txt) lists them all. Put the file and its `.sha256` in one folder and run the check there. This example checks the extension: ```bash curl -fLO https://thinqit.ai/downloads/thinqit-dex-vscode-0.1.1.vsix.sha256 sha256sum -c thinqit-dex-vscode-0.1.1.vsix.sha256 # Linux shasum -a 256 -c thinqit-dex-vscode-0.1.1.vsix.sha256 # macOS ``` Both print `OK` when the file is intact. In PowerShell on Windows, these two lines must print the same hash: ```powershell (Get-FileHash .\thinqit-dex-vscode-0.1.1.vsix -Algorithm SHA256).Hash.ToLower() (Get-Content .\thinqit-dex-vscode-0.1.1.vsix.sha256).Split(" ")[0] ``` ### Licence and executables All four clients are open source under the Apache-2.0 licence. Every package holds its LICENSE and NOTICE files, and both are also here: [LICENSE.txt](/downloads/LICENSE.txt) and [NOTICE.txt](/downloads/NOTICE.txt). The CLI's single executables are not offered for download, because each one is about 100 MB. Use the npm package with Node.js 18 or newer instead. ## Configuration | Setting | Python | TypeScript | Environment variable | Default | | --- | --- | --- | --- | --- | | API key | `api_key` | `apiKey` | `DEX_API_KEY` | None: required | | Base URL | `base_url` | `baseUrl` | `DEX_BASE_URL` | `https://api.thinqit.ai` | | Timeout per attempt | `timeout` (seconds) | `timeout` (milliseconds) | | 30 seconds | | Retries | `max_retries` | `maxRetries` | | 3 | | Extra headers | `default_headers` | `defaultHeaders` | | None | A value passed to the constructor wins over the environment variable. To run against the mock server, set `DEX_BASE_URL=http://127.0.0.1:4010`. The base URL must use `https://`, so your key is never sent in clear text; `http://` is accepted only for `localhost`, `127.0.0.1` and `[::1]`, and the SDKs, the CLI and the VS Code extension refuse any other `http://` address before sending a request. ## A first call This sends the support ticket from the [Quickstart](/docs/quickstart/#make-your-first-call) and acts only on an answer that did not abstain. ```python tab="Python" from thinqit_dex import Client, pick, rate, check client = Client() # reads DEX_API_KEY decision = client.decide( state={ "ticket": { "channel": "email", "subject": "Charged twice, still no answer", "body": "This is the third time I am writing. You charged my card twice " "for order 4471 and nobody replies. Fix it today or I cancel my subscription.", } }, questions={ "team": pick( "Which team should handle {{ticket.body}}?", { "billing": "Payments, refunds and invoices", "technical": "Bugs, outages and sign-in problems", "shipping": "Delivery and returns", "other": None, }, min_confidence=0.5, ), "urgency": rate( "How fast must we reply to {{ticket.body}}?", ["Can wait", "This week", "Within two working days", "Today"], ), "angry": check("Is the writer of {{ticket.body}} angry?", min_confidence=0.6), }, ) team = decision.pick("team") if not team.abstained: print(team.choice) ``` ```ts tab="TypeScript" import { Client, pick, rate, check } from "@thinqit/dex"; const client = new Client(); // reads DEX_API_KEY const decision = await client.decide({ state: { ticket: { channel: "email", subject: "Charged twice, still no answer", body: "This is the third time I am writing. You charged my card twice for order 4471 and nobody replies. Fix it today or I cancel my subscription.", }, }, questions: { team: pick( "Which team should handle {{ticket.body}}?", { billing: "Payments, refunds and invoices", technical: "Bugs, outages and sign-in problems", shipping: "Delivery and returns", other: null, }, { min_confidence: 0.5 }, ), urgency: rate("How fast must we reply to {{ticket.body}}?", [ "Can wait", "This week", "Within two working days", "Today", ]), angry: check("Is the writer of {{ticket.body}} angry?", { min_confidence: 0.6 }), }, }); if (!decision.answers.team.abstained) { console.log(decision.answers.team.choice); } ``` ### What the SDKs cover | Operation | Python | TypeScript | | --- | --- | --- | | [decide](/docs/api/decide/) | `client.decide(state, questions, *, model=None, fallback=None, idempotency_key=None, ...)` | `client.decide(request, options?)` | | [listModels](/docs/api/listModels/) | `client.list_models()` | `client.listModels()` | | [getCurrentKey](/docs/api/getCurrentKey/) | `client.get_key()` | `client.getKey()` | | [getUsage](/docs/api/getUsage/) | `client.get_usage(start, end, *, granularity=None, key_id=None)` | `client.getUsage({ start, end, granularity?, key_id? })` | | [getBalance](/docs/api/getBalance/) | `client.get_balance()` | `client.getBalance()` | Every method takes a client request id (`client_request_id` in Python, `clientRequestId` in TypeScript), which the API echoes in `x-client-request-id`. See [Headers](/docs/reference/headers/#request-ids). The console routes are not part of the SDKs. Python also has `AsyncClient`, with the same arguments and `await`-able methods: ```python from thinqit_dex import AsyncClient async with AsyncClient() as client: decision = await client.decide(state=state, questions=questions) ``` ### Builders and typed answers - `pick(instructions, options, ...)`, `rate(instructions, levels, ...)` and `check(instructions, ...)` return plain question objects. `rate` is in beta: see [Questions](/docs/concepts/questions/#rate). Each also takes `criteria` and `min_confidence`: as keyword arguments in Python, and in a final `{ criteria, min_confidence }` object in TypeScript. - In Python, `decision.pick(id)`, `decision.rate(id)` and `decision.check(id)` return typed answers. A wrong id raises `KeyError`, and a wrong type raises `TypeError`. - In TypeScript, `decision.answers.` is typed from your request. The labels of a pick become a union type for `choice` and `probabilities`. - Every answer type lists `abstained` first. Check it before you use an answer. See [Abstention](/docs/concepts/abstention/). - `ref(...)` and `escape_braces` (`escapeBraces` in TypeScript) build [field references](/docs/concepts/field-references/#sdk-helpers). ### Response details Every result has an `http` property with the status, the request id, your client request id, the idempotency key, whether it was a replay, the rate limit headers, how many retries it took, and all headers. Python uses snake_case names, such as `result.http.request_id`, and TypeScript uses camelCase, such as `result.http.requestId`. ## Retries and idempotency The SDKs retry for you: - **What they retry:** HTTP 429, 500, 502, 503 and 504, 409 `idempotency_in_progress`, connection errors and timeouts. Never any other 4xx, and never 429 `test_daily_quota`, which lasts until 00:00 UTC. A load balancer can return 502 or 504 during a restart, which is why those are retried although the API itself does not send them. - **How often:** up to `max_retries` retries (3 by default), so at most 4 attempts. - **How long they wait:** the `retry-after` header when there is one. Otherwise a random delay between 0 and `min(8, 0.25 * 2^n)` seconds before retry `n`, counting from 0. - **No double charges:** every `decide` call sends an `idempotency-key`, yours or a new UUID v4, and reuses it on every retry. A retry can never charge twice. Read-only calls send none. See [Idempotency](/docs/reference/idempotency/#sdks). Set `max_retries=0` (or `maxRetries: 0`) on one call to turn retries off for it. ## Errors Each error `type` has its own class, such as `RateLimitError` for a 429 and `RequestValidationError` for a 422. All of them carry `status`, `type`, `code`, `message`, `param`, `request_id` and `retry_after`. The full table is in [Errors](/docs/reference/errors/#sdk-error-classes). The SDKs are built and tested never to write your API key to logs, error messages or object representations. ## The CLI Install the CLI from [Downloads](#downloads) until it is on npm. The command is `dex`. ```bash dex login # checks the key's checksum, then stores it in the OS keychain dex whoami # which key and account you are using (GET /v1/key) dex decide --file request.json # sends a request file, such as the one from the Quickstart dex models # lists models dex usage --since 7d # shows your usage dex balance # shows your balance ``` - `dex decide` also takes inline flags for quick checks, such as `--state @ticket.txt` for a state read from a file. - Output is a table by default, or JSON with `--json`. - The exit code tells scripts what happened: 0 success, 2 validation, 3 authentication, 4 balance, 5 rate limit, 6 unavailable. - `dex login` reads the key from standard input, so it never appears in your shell history. It refuses a mistyped key offline, from the key's [checksum](/docs/reference/authentication/#key-format-and-checksum). - `dex decide --file` keeps the key order of your file exactly, rejects duplicate keys, and removes a top-level `$schema` line before sending. ## VS Code extension The VS Code extension validates request files against the contract, runs them from the editor and shows the answers inline. Its status bar shows which key you use, from `GET /v1/key`. Install it from [Downloads](#downloads) until it is on the Marketplace. ## JSON schema for request files The request schema is published at https://thinqit.ai/schemas/dex-request.schema.json, generated from the API contract. Add `"$schema": "https://thinqit.ai/schemas/dex-request.schema.json"` at the top of a request file to get completion and validation in any editor that reads JSON schemas. The CLI and the VS Code extension remove `$schema` before sending. The API itself rejects it as an unknown field, so leave it out of bodies you send with curl or an SDK. --- # Changelog > Changes to the website, the documentation, the API contract and the model versions, newest first. ## 26 September 2026, dex-1.0.1 - New exact version `dex-1.0.1`: text you send can no longer act as prompt structure. A `` or `` tag inside the state, and markup such as ``, `` or `<|im_end|>` anywhere in a request, is read as plain text, and the prompt reminds the model that the state is data. Same model and engine as `dex-1.0.0`, new calibration `cal-20260926-1`. See [Models and versions](/docs/concepts/models/#versions) and [State](/docs/concepts/state/#the-state-is-data). - The alias `dex-1` now points at `dex-1.0.1`. Requests without a `model`, or with `"model": "dex-1"`, are answered by it. - `dex-1.0.0` stays available unchanged for at least 90 days: a request pinned to it gets exactly the same answers as before. - Lower request limits, for every version: at most 16,384 billable tokens per request (was 32,768), and at most 4,096 for all questions of a request together. Larger requests could not be answered on the GPU in time. Both give 413 `request_too_long`. See [Limits](/docs/reference/limits/). - Stricter request validation, for every version. Requests that can only confuse the model are now refused before anything is charged. A state, `instructions`, `criteria` (string or list item) or option description made only of whitespace, and an empty object state `{}` (as `[]` already was), give 422 `invalid_value`. A `rate` level with whitespace at either end or a line break gives 422 `invalid_value`, with the level in `param`. Two levels that differ only in case or Unicode normalization give 422 `invalid_levels`, and two option labels that do, such as `Billing` and `billing`, give 422 `duplicate_label`; a label repeated exactly stays 400 `duplicate_key`. A `\u` escape that is half of a surrogate pair, such as `"\ud800"` alone, gives 400 `invalid_json`. Requests that were accepted before and still are get the same answers. See [Errors](/docs/reference/errors/#codes-in-detail). ## 26 September 2026 - The SDKs, the CLI and the VS Code extension are at 0.1.1 in [Downloads](/docs/reference/sdks/#downloads). The base URL must use `https://`, so your key is never sent in clear text; `http://` works only for `localhost`, `127.0.0.1` and `[::1]`, for the mock server. The 0.1.0 files were rebuilt once from newer code under the same name; from now on every change gets a new version. - The website runs no inline script except the theme switch, which its content security policy allows by hash, and `strict-transport-security` covers the subdomains of thinqit.ai. - Sign-up and API keys are open: sign in to the [console](/console/) with your email address and create a free test key. Paid top-ups open soon. - Interim downloads of the Python and TypeScript SDKs, the CLI and the VS Code extension, until they are on PyPI, npm and the Marketplace. See [Downloads](/docs/reference/sdks/#downloads). - The mock server of the API contract answers the [Quickstart](/docs/quickstart/#try-it-today-with-a-mock-server) ticket by default, so the SDK quickstarts run against it. - TypeScript SDK: `parseRequest(text)` reads a stored request without changing the order of labels such as "5", "4", "3". - Console sign-in: a wrong code answers `invalid_code`, and after 5 wrong tries `code_attempts_exceeded`. When an address has had 5 codes in an hour, `retry-after` gives the real wait, up to an hour. See [Errors](/docs/reference/errors/#codes-in-detail). - `test_daily_quota` gives the time until 00:00 UTC in `retry-after`, and the SDKs no longer retry it. A request that fails with 500 or 503 gives back its test quota and tokens per minute. See [Rate limits](/docs/reference/rate-limits/#when-you-hit-a-limit). - A known path called with a method it does not have answers 405 `method_not_allowed`, with an `allow` header. - The CLI and the VS Code extension link to the console at https://thinqit.ai/console/. ## 25 September 2026, night - New pages: the [status page](/status/), which checks the API, the GPU path and the fallback from your browser every 30 seconds, and the console at [/console/](/console/): sign in with an email code, create and revoke API keys, see usage per day and per key, run requests in a test-mode playground, and top up (payments open with the API). - The website is simpler: Home, [Use cases](/use-cases/), [Pricing](/pricing/) and the docs. The race moved to the [home page](/#race), the comparison and the cost chart to the [pricing page](/pricing/#jev), and this changelog into the docs. - Measured quality of `dex-1.0.0` (calibration `cal-20260925-1`) is published on the [calibration page](/docs/calibration/): agreement overall and per language, and agreement, calibration error and a reliability curve per question type. - `rate` questions are labelled beta: their agreement and calibration error miss our bars. See [Quality bars](/docs/concepts/calibration/#quality-bars). - [Calibration](/docs/concepts/calibration/) now describes the evaluation set this version was measured on, and its known limits. ## 25 September 2026, evening - Launch price set: EUR 0.05 per million input tokens, flat at every volume. Output stays free. See [Pricing](/pricing/). - New page: [Use cases](/use-cases/), in English and [Dutch](/nl/use-cases/). When to use Dex instead of an LLM, ten business cases, and the cases it does not suit. - Imprint added to the footer and the [legal pages](/legal/#imprint): Thinqit B.V., Vanadiumweg 25, 3812PX Amersfoort, the Netherlands. - The SDKs, the CLI and the VS Code extension will be published under the Apache-2.0 licence. ## 25 September 2026, contract round 1 - New route [`GET /v1/key`](/docs/api/getCurrentKey/): which key and account you are using, with any key and no scope. See [Authentication and API keys](/docs/reference/authentication/), which also documents the key format and its checksum. - New error codes: 400 `duplicate_key` for a repeated JSON key, and 422 `invalid_value` for a value of the right type that is not allowed. [Errors](/docs/reference/errors/#codes-in-detail) now says when every 400 and 422 code applies. - `x-client-request-id` is accepted and echoed on every public route. - Test keys get 40,000 tokens per minute, so the largest request always fits. Every public route counts one request against requests per minute. See [Rate limits](/docs/reference/rate-limits/). - An idempotent replay returns the original response, with its original `id`, `created` and `usage`, and charges nothing. A 429 or 402 keeps no idempotency record. See [Idempotency](/docs/reference/idempotency/). - [Determinism](/docs/concepts/determinism/) is stated for identical request bytes on the same GPU version. v1 does not promise question isolation; it is on the roadmap. - The request JSON schema for editors is published at `https://thinqit.ai/schemas/dex-request.schema.json`. - Moderation traffic should send `"fallback": "never"`. See [Fallback and served_by](/docs/concepts/fallback/#moderation-traffic). ## 25 September 2026 - Published the website and the documentation. - Published the draft of the v1 API contract. You can download it as `https://thinqit.ai/openapi.yaml` and read it in the [API reference](/docs/api/decide/). - The API is not open yet. API keys, the console at https://thinqit.ai/console/ and the SDK packages arrive at launch. Until then you can run a mock of the API from the contract: see the [Quickstart](/docs/quickstart/#try-it-today-with-a-mock-server). --- # API reference Base URL: `https://api.thinqit.ai`. Contract version 1.0.0, rendered from [openapi.yaml](https://thinqit.ai/openapi.yaml). ## Decisions Ask typed questions about a state. - [Answer typed questions about a state](https://thinqit.ai/docs/api/decide/): `POST /v1/decide` ## Models Aliases and exact model versions, with their limits and published quality. - [List model aliases and exact versions](https://thinqit.ai/docs/api/listModels/): `GET /v1/models` ## Account The calling API key, and the usage and balance of its account. - [The API key making this call](https://thinqit.ai/docs/api/getCurrentKey/): `GET /v1/key` - [Usage for one API key over a date range](https://thinqit.ai/docs/api/getUsage/): `GET /v1/usage` - [Current prepaid balance and price tier](https://thinqit.ai/docs/api/getBalance/): `GET /v1/balance` ## Console auth Sign-in for the web console. Console routes use a session cookie, not an API key. - [Email a one-time sign-in code](https://thinqit.ai/docs/api/requestSignInCode/): `POST /v1/console/auth/email-code` - [Exchange a sign-in code for a session](https://thinqit.ai/docs/api/createSession/): `POST /v1/console/auth/session` - [Sign out](https://thinqit.ai/docs/api/deleteSession/): `DELETE /v1/console/auth/session` ## Console keys Create, list, change and revoke API keys. - [List API keys](https://thinqit.ai/docs/api/listKeys/): `GET /v1/console/keys` - [Create an API key](https://thinqit.ai/docs/api/createKey/): `POST /v1/console/keys` - [Read one API key](https://thinqit.ai/docs/api/getKey/): `GET /v1/console/keys/{key_id}` - [Rename a key, change its scopes or lower its limits](https://thinqit.ai/docs/api/updateKey/): `PATCH /v1/console/keys/{key_id}` - [Revoke an API key](https://thinqit.ai/docs/api/revokeKey/): `DELETE /v1/console/keys/{key_id}` ## Console billing Account profile and prepaid balance top-ups through Stripe Checkout. - [Account profile, balance and settings](https://thinqit.ai/docs/api/getAccount/): `GET /v1/console/account` - [Start a Stripe Checkout top-up](https://thinqit.ai/docs/api/createCheckoutSession/): `POST /v1/console/checkout-sessions` ## Console usage Account-wide usage series for the console charts. - [Account usage as a time series](https://thinqit.ai/docs/api/getUsageSeries/): `GET /v1/console/usage/series` ## Console playground Run decisions from the console in test mode. - [Run a decision from the console in test mode](https://thinqit.ai/docs/api/playgroundDecide/): `POST /v1/console/playground/decide` --- # Answer typed questions about a state `POST /v1/decide` (operation id `decide`) Answers every question against the shared state in one call. The state is billed once. Requests to an exact GPU model version (for example `dex-1.0.0`) or with `fallback: "never"` are never served by the fallback path; send `fallback: "never"` for content moderation, because the fallback's content filters block that content. A body that repeats a key in any JSON object is rejected with 400 `duplicate_key`. See `docs/spec/dex-v1.md` sections 5 to 7. Authentication: API key as `authorization: Bearer `, scope `decide`. ## Parameters | Name | In | Type | Required | Description | | --- | --- | --- | --- | --- | | `idempotency-key` | header | string | no | Makes retries safe. Within 24 hours, a repeat with the same key and the same body is never charged again. It answers with `idempotent-replayed: true` and the original response: the same `id`, `created`, `model`, answers and `usage`, including the original charge fields. The same key with a different body answers 409. (1 to 255 characters; pattern ^[\x21-\x7E]+$) | | `x-client-request-id` | header | string | no | Your own correlation id, accepted on every public operation. Logged with the request metadata and echoed in the response. A malformed value answers 400 `invalid_header`. (1 to 128 characters; pattern ^[\x21-\x7E]+$) | ## Request body | Field | Type | Required | Description | | --- | --- | --- | --- | | `model` | ModelName | no | An alias such as `dex-1` or an exact version such as `dex-1.0.0`. (default "dex-1"; pattern ^dex-[0-9]+(\.[0-9]+\.[0-9]+)?$) | | `state` | State | yes | The material the questions are about. A string, or JSON whose fields questions can point at with `{{path}}` references. A string of only whitespace, an empty object and an empty array are rejected (empty objects and arrays inside the state are fine). At most 16,384 tokens after rendering. (string: 1 to 65,536 characters, pattern \S; object: at least 1 entries; array of any: at least 1 items) | | `questions` | object | yes | Question id to question. Ids are for your code only and never reach the model. (1 to 32 entries; keys match ^[A-Za-z][A-Za-z0-9_-]{0,63}$) | | `questions.` | Question | no | | | **When `type` is `pick`** | PickQuestion | | | | `questions..type` | string | yes | (always "pick") | | `questions..instructions` | Instructions | yes | What to decide; a string of only whitespace is rejected. May contain `{{path}}` references to state fields, such as `{{ticket.body}}`. (1 to 4,000 characters; pattern \S) | | `questions..criteria` | Criteria | no | Extra rules or definitions the model must apply. May contain `{{path}}` references. A string of only whitespace is rejected. `null` is the same as leaving the field out. (string: 1 to 4,000 characters, pattern \S; array of string: 1 to 20 items) | | `questions..options` | object | yes | Label to description (or null). Key order is kept and shown to the model in that order. Labels have no leading or trailing whitespace and are unique after Unicode NFC normalisation and lower-casing (`Billing` and `billing` are the same label). A description of only whitespace is rejected. (1 to 255 entries; keys match ^\S(.*\S)?$, 1 to 100 characters; values: string \| null, 1 to 1,000 characters, pattern \S) | | `questions..options.