# Pattern: pin versions in CI

> Tests that send pinned requests and expect the exact recorded answers, a check that warns before a pinned version retires, and a CI job that runs both with a free test key.

On the GPU path, identical request bytes sent to the same exact version return byte-identical answers, every time. That turns your Dex requests into ordinary test fixtures: record the answers once, pin the version, and let CI fail when anything changes. See [Determinism](/docs/concepts/determinism/).

## The rules

- **Pin the exact version** in every test request: `"model": "dex-1.0.1"`, exactly as `GET /v1/models` lists it. An exact version never goes to the hosted fallback, so a test can never be answered by a path that is not deterministic.
- **Send the same bytes.** Keep each test request as a file and send the file's text as it is. Two requests that differ only in whitespace or key order are different requests.
- **Compare `answers`, `model`, `calibration` and the token counts of `usage`.** `id` and `created` differ between calls, and on a live key the split between `allowance_tokens` and `paid_tokens` depends on the week's allowance.
- **Raw HTTP, on purpose.** The tests send the file's bytes with plain HTTP, so the SDK's retries do not apply: rerun the job after a `429` or `503`, or wrap the call in the retry loop of [Retries, timeouts and fallbacks](/docs/cookbook/retries-and-fallbacks/#with-plain-http).
- **Use a test key.** Test calls are free up to 250,000 tokens a day per account and always run on the GPU path. Store the key as a CI secret.
- **Watch the lifecycle.** A replaced version stays available for at least 90 days and `GET /v1/models` shows its `retires_at`. Move the pin on purpose: record new answers on the new version and review the difference like any other change.

## Record once

Put each test request in `tests/dex/<name>.request.json`, with `"model"` pinned. The request of the [support ticket recipe](/docs/cookbook/support-routing/) with a pin is a good first one. Then record the answers:

```python
# record.py: python record.py  (writes tests/dex/<name>.expected.json next to each request)
import json, os, pathlib
import httpx

for req in sorted(pathlib.Path("tests/dex").glob("*.request.json")):
    body = req.read_bytes()  # the exact bytes, never re-serialised
    r = httpx.post(
        "https://api.thinqit.ai/v1/decide",
        content=body,
        headers={"authorization": f"Bearer {os.environ['DEX_API_KEY']}", "content-type": "application/json"},
        timeout=30,
    )
    r.raise_for_status()
    d = r.json()
    assert d["served_by"] == "gpu", "pinned requests are always served by the GPU path"
    keep = {k: d[k] for k in ("model", "calibration", "answers")}
    keep["usage"] = {k: d["usage"][k] for k in ("input_tokens", "state_tokens", "question_tokens")}
    req.with_name(req.name.replace(".request.json", ".expected.json")).write_text(json.dumps(keep, indent=2) + "\n")
    print(req.name, d["model"], d["usage"]["input_tokens"], "tokens")
```

Commit the `.expected.json` files.

## Test on every change

```python tab="Python (pytest)"
# tests/test_dex_pinned.py
import json, os, pathlib
from datetime import datetime, timedelta, timezone

import httpx
import pytest

API = "https://api.thinqit.ai"
HEADERS = {"authorization": f"Bearer {os.environ['DEX_API_KEY']}", "content-type": "application/json"}
CASES = sorted(pathlib.Path("tests/dex").glob("*.request.json"))


@pytest.mark.parametrize("req", CASES, ids=lambda p: p.name)
def test_pinned_answers_do_not_change(req: pathlib.Path) -> None:
    expected = json.loads(req.with_name(req.name.replace(".request.json", ".expected.json")).read_text())
    r = httpx.post(f"{API}/v1/decide", content=req.read_bytes(), headers=HEADERS, timeout=30)
    r.raise_for_status()
    got = r.json()
    got["usage"] = {k: got["usage"][k] for k in expected["usage"]}  # token counts only
    assert {k: got[k] for k in expected} == expected


def test_pinned_versions_are_not_about_to_retire() -> None:
    pins = {json.loads(p.read_text())["model"] for p in CASES}
    models = {m["id"]: m for m in httpx.get(f"{API}/v1/models", headers=HEADERS, timeout=30).json()["data"]}
    soon = datetime.now(timezone.utc) + timedelta(days=30)
    for pin in pins:
        assert pin in models, f"{pin} is not listed any more"
        m = models[pin]
        assert m["status"] == "active", f"{pin} is {m['status']}"
        if m["retires_at"]:
            assert datetime.fromisoformat(m["retires_at"].replace("Z", "+00:00")) > soon, f"{pin} retires {m['retires_at']}"
```

```ts tab="TypeScript (node:test)"
// tests/dex-pinned.test.mts: node --test "tests/*.test.mts"  (Node 22.18 or newer)
import { test } from "node:test";
import assert from "node:assert/strict";
import { readFile, readdir } from "node:fs/promises";

const API = "https://api.thinqit.ai";
const headers = { authorization: `Bearer ${process.env.DEX_API_KEY}`, "content-type": "application/json" };
const dir = "tests/dex";
const cases = (await readdir(dir)).filter((f) => f.endsWith(".request.json")).sort();

for (const name of cases) {
  test(`pinned answers do not change: ${name}`, async () => {
    const body = await readFile(`${dir}/${name}`, "utf8"); // sent as it is: never JSON.parse and stringify
    const expected = JSON.parse(await readFile(`${dir}/${name.replace(".request.json", ".expected.json")}`, "utf8"));
    const res = await fetch(`${API}/v1/decide`, { method: "POST", headers, body });
    assert.equal(res.status, 200, await res.clone().text());
    const got = await res.json();
    got.usage = Object.fromEntries(Object.keys(expected.usage).map((k) => [k, got.usage[k]])); // token counts only
    for (const k of Object.keys(expected)) assert.deepEqual(got[k], expected[k], k);
  });
}

test("pinned versions are not about to retire", async () => {
  const pins = new Set<string>();
  for (const name of cases) pins.add(JSON.parse(await readFile(`${dir}/${name}`, "utf8")).model);
  const models = (await (await fetch(`${API}/v1/models`, { headers })).json()).data as Array<{
    id: string;
    status: string;
    retires_at: string | null;
  }>;
  const soon = Date.now() + 30 * 24 * 3600 * 1000;
  for (const pin of pins) {
    const m = models.find((x) => x.id === pin);
    assert.ok(m, `${pin} is not listed any more`);
    assert.equal(m.status, "active", `${pin} is ${m.status}`);
    if (m.retires_at) assert.ok(Date.parse(m.retires_at) > soon, `${pin} retires ${m.retires_at}`);
  }
});
```

Comparing parsed JSON with `deepEqual` is exact here: the answers are compared value by value, and a pick's labels are keys of an object, so their order does not matter to the comparison.

## Run it in CI

Store a test key as a CI secret and run the tests with it in `DEX_API_KEY`. In GitHub Actions, these are the steps of a job that has checked out the repository and set up Python, with the key stored as the repository secret `DEX_TEST_KEY`:

```yaml
      - run: pip install pytest httpx
      - run: pytest tests/test_dex_pinned.py
        env:
          DEX_API_KEY: ${{ secrets.DEX_TEST_KEY }}
```

The TypeScript version runs with `node --test "tests/*.test.mts"` and the same variable, on Node 22.18 or newer, which runs TypeScript files as they are. On Node 20, install `tsx` and run `node --import tsx --test tests/dex-pinned.test.mts`.

## When a test fails

- **Answers differ on the same version:** check that the request file changed only where you meant it to (whitespace and key order count). If it did not change, write to hello@thinqit.io with the request id: that would break the determinism guarantee.
- **`404 model_retired` or the retirement check fails:** move the pin. Record the answers on the new version with `record.py`, review the difference, and commit the new files together.
- **`503 no_capacity`:** the pinned version had no GPU capacity for a moment. Retry the job; do not switch a pinned test to the alias.

For production code, keep the alias `dex-1` (or pin as well and move deliberately) and keep the answers your code acts on above a `min_confidence`: a new version can move borderline answers.