# Confidence

> How the confidence value of each answer type is computed, what it does not promise, and how numbers are rounded.

Every answer has a `confidence` between 0 and 1. It says how concentrated the answer's probability distribution is: 1 when all probability sits on one answer, lower as it spreads out. You use it to decide when to trust an answer, usually through [abstention](/docs/concepts/abstention/).

## From logits to probabilities

For each question, the model produces one raw score (a logit) per possible answer: one per option for a `pick`, one per level for a `rate`, and a yes and a no score for a `check`. Dex divides these scores by a temperature `T` for the question type, then turns them into probabilities. The temperatures come from the model version's [calibration](/docs/concepts/calibration/), so there is one `T` each for pick, rate and check.

The formulas below use `z` for the logits and `p` for the probabilities.

## pick

The probabilities are a softmax over the options. The choice is the most probable option. Confidence is the gap between the two largest probabilities.

```text
p_i        = softmax(z_i / T_pick)      over the n options
choice     = argmax p                    ties go to the earlier option
confidence = p_(1) - p_(2)              largest minus second largest
confidence = 1                           when there is only one option
```

In the Dutch support example in the contract, `garantie` has 0.6423 and `reparatie` has 0.3287, so confidence is 0.6423 minus 0.3287, which is 0.3136. Two options are close, and confidence is low.

## rate

The probabilities are a softmax over the levels. The rating is their probability-weighted index. Confidence measures how tightly the probability clusters around the rating, relative to the widest possible spread.

```text
p_j        = softmax(z_j / T_rate)      over the k levels
rating     = sum_j j * p_j
sigma      = sqrt(sum_j p_j * (j - rating)^2)
confidence = 1 - sigma / ((k - 1) / 2)
```

Confidence is 1 when all probability sits on one level. It is 0 when the probability splits evenly between the lowest and the highest level, the widest spread possible.

## check

The probability of yes is a sigmoid of the gap between the yes and no scores. Confidence is the distance from 50/50, scaled to 0 to 1.

```text
probability = sigmoid((z_yes - z_no) / T_check)
confidence  = |2 * probability - 1|
```

This is the same gap as a `pick` with two options. A probability of 0.5 has confidence 0, and a probability of 0.75 has confidence 0.5.

It helps to translate a threshold back into probabilities. A check reaches confidence 0.6 only when the probability is at least 0.8 or at most 0.2.

## What confidence is not

- **Not a correctness guarantee.** A confident answer can be wrong. Confidence only describes the shape of the distribution. [Calibration](/docs/concepts/calibration/) is what makes the probabilities mean what they say.
- **Not one scale across types.** Each type has its own formula, so 0.5 on a `rate` does not mean the same as 0.5 on a `pick`. Choose `min_confidence` per question, not one number for all.
- **Not a probability.** For a check, `probability` is the probability of yes, and `confidence` is derived from it. Use `probability` when you need the chance that something holds.

## Rounding

Dex rounds numbers the same way every time, so the same inputs always give the same bytes.

- **Probabilities** are rounded to 4 decimals with the largest-remainder method, so each distribution sums to exactly 1.0000. Equal remainders go to the earlier option or level.
- **`rating`, `confidence` and a check's `probability`** are computed from the unrounded probabilities, then rounded half to even to 4 decimals.
- **`choice` and `abstained`** are decided on the unrounded values.
- **Numbers** are written in plain decimal notation, never with an exponent.

Because confidence comes from unrounded values, recomputing it from the rounded numbers can differ in the last digit. In the Dutch support example, the check has `probability` 0.8732 and `confidence` 0.7465, while two times 0.8732 minus 1 gives 0.7464.