> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.milliseconds.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.milliseconds.ai/_mcp/server.

# How it works

`decision-machine-1` is not one large model. It is two small encoder models behind one API, plus a scheduler that keeps them busy.

Each call runs one pass over your text. It uses no sampling loop, no retry chain and no prompt parser. That is where the speed comes from.

## The two models

| Model          | Job                                                     | Capabilities                                                                                                                                                        |
| -------------- | ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| The classifier | Scores every label you send in one pass over the text   | [Yes / no](/capabilities/yes-no), [Classify](/capabilities/classify), [Rate](/capabilities/rate), and the tool pick in [function calling](/openai/function-calling) |
| The extractor  | Finds the best span in the text for each label you send | [Answer](/capabilities/answer), [Extract](/capabilities/extract), [Entities](/capabilities/entities), [Verify](/capabilities/verify)                                |

The classifier scores labels. The extractor finds spans. Every capability maps to one of those two jobs.

> **Note**
>
> **Labels are the interface.** The classifier reads your label text and scores it against the input. The words `yes` and `no` carry no meaning for it. See [Writing good statements and labels](/concepts/writing-good-questions).

### What the classifier does

The classifier scores every label independently in the same pass, so adding labels adds their length to the input and nothing else.

The API then normalises those scores. [Decisions and probabilities](/concepts/decisions-and-probabilities) gives the exact formulas.

### What the extractor does

The extractor finds the best span for each label you supply. `extract` sends your JSON Schema as a structure. `answer` sends each question as a label, plus one decoy label named `other`. The decoy sharpens the spans, and the API discards it. `entities` sends each type as a label. `verify` checks one field against one value.

The extractor is multilingual. The classifier performs best in English. See [Languages](/concepts/languages).

## One pass per call

Every capability call does the same three things:

#### The API builds the labels

The API turns your request into labels or structures. It validates the field limits first, so bad input fails in under 0.2 s.

#### The scheduler leases a slot

A single scheduler holds every inference slot. It leases one call per slot and queues the rest. It opens the circuit when a box fails.

#### The model runs once

The model reads the text once and returns scores or spans. The API normalises, rounds to three decimals, and returns typed JSON.

Inference runs on a pool of GPUs, with CPU boxes as fallback. The scheduler tries a GPU slot first. Each GPU serves several calls in flight and batches classifier calls into one forward pass.

## Chunking at 2,000 characters

Both models slow down past roughly 500 tokens. The pipeline splits any text over 2,000 characters on whitespace, so every chunk stays in the linear regime.

| Capability                   | Behaviour over 2,000 characters                                                                                                                   |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| `yes-no`, `classify`, `rate` | A label's score is its **max over chunks**. A claim that holds only in the last paragraph still scores high.                                      |
| `extract`                    | Per-chunk records merge. A single-record structure folds into one record, first non-empty value per field. A multi-record structure concatenates. |
| `entities`, `verify`         | The extractor runs overlapping windows and remaps the offsets back to your text.                                                                  |

A single over-long token becomes its own chunk. `extract` also collapses runs of two or more spaces to `" | "` first, so table cells read as distinct spans.

Cost is linear in characters. [Long text and chunking](/concepts/long-text) covers the practical guidance on pre-splitting.

## Measured latency

Two sets of numbers. The first is the model time per call on the GPU pool, from the `x-inference-ms` header, short inputs.

| Call group                                                   | Model time |
| ------------------------------------------------------------ | ---------- |
| `classify` / `yes-no` / `rate` (classifier, any label count) | \~54 ms    |
| `entities` / `answer` / `extract` / `verify` (extractor)     | \~36 ms    |
| `classify-tree`, 2 levels                                    | \~108 ms   |

Label count does not move the classifier time. Extra `statements`, `questions` or `texts` ride in the same batch.

The second set is end-to-end wall clock, measured from a laptop against production on 2026-09-18, median of 7 runs. Inputs were 51 to 199 characters. These numbers include TLS, internet transit and the API hop. The last column is the `x-inference-ms` header: the model time alone.

| Call                             | Observed    | Model time |
| -------------------------------- | ----------- | ---------- |
| `yes-no`, one statement          | 0.42 s      | 54 ms      |
| `yes-no`, 2 statements           | 0.49 s      | 107 ms     |
| `yes-no`, 2 texts x 2 statements | 0.49 s      | 214 ms     |
| `classify`, 3 labels             | 0.43 s      | 54 ms      |
| `classify-tree`, 2 levels        | 0.77 s      | 108 ms     |
| `rate`, 4 levels                 | 0.42 s      | 53 ms      |
| `answer`, 1 question             | 0.40 s      | 36 ms      |
| `answer`, 3 questions            | 0.40 s      | 36 ms      |
| `entities`, 3 types              | 0.42 s      | 36 ms      |
| `verify`                         | 0.40 s      | 36 ms      |
| `extract`, 9 properties          | 0.41 s      | 44 ms      |
| `extract`, 2 texts               | 0.44 s      | 154 ms     |
| chat `json_schema`               | 0.39 s      | 37 ms      |
| chat `tools`, 2 tools            | 0.72 s      | 88 ms      |
| chat forced `tool_choice`        | 0.40 s      | 36 ms      |
| Every 400 / 404                  | under 0.1 s |            |

Every single-pass call lands near 0.4 s. The model takes 36 to 54 ms of that; the rest is the network and the API hop.

> **Tip**
>
> **Batch the cheap axis.** Extra `statements` or `questions` ride in one runner call and add only their own length to the billed input. Extra `texts` cost one call each. See [Batching](/capabilities/batching).

## What you can measure yourself

Every capability response carries the input size and the model processing time in headers. Use them to predict cost, to size your batches and to watch latency.

**`curl`**

```bash title="curl"
curl -sS -D - -o /dev/null -X POST https://api.milliseconds.ai/v1/decision-machine-1/classify \
  -H "Content-Type: application/json" \
  -H "authorization: Bearer test_sk-..." \
  -d '{
    "text": "My card was charged twice this month.",
    "labels": ["billing", "bug", "feature request"]
  }' | grep -iE "x-input|x-inference"
```

**`Python`**

```python title="Python"
import requests

r = requests.post(
    "https://api.milliseconds.ai/v1/decision-machine-1/classify",
    headers={"authorization": "Bearer test_sk-..."},
    json={
        "text": "My card was charged twice this month.",
        "labels": ["billing", "bug", "feature request"],
    },
)

print(r.headers["x-input-chars"], r.headers["x-input-tokens"], r.headers["x-inference-ms"])
```

**`TypeScript`**

```typescript title="TypeScript"
const r = await fetch(
  "https://api.milliseconds.ai/v1/decision-machine-1/classify",
  {
    method: "POST",
    headers: { "Content-Type": "application/json", authorization: "Bearer test_sk-..." },
    body: JSON.stringify({
      text: "My card was charged twice this month.",
      labels: ["billing", "bug", "feature request"],
    }),
  },
);

console.log(
  r.headers.get("x-input-chars"),
  r.headers.get("x-input-tokens"),
  r.headers.get("x-inference-ms"),
);
```

```
x-input-chars: 37
x-input-tokens: 10
x-inference-ms: 412
```

`x-input-tokens` is the input tokens billed for this call. `x-input-chars` is the number of input characters, and it sums every item for `texts`. `x-inference-ms` is the model processing time for this request, summed over the model calls it made. The inference service measures it around the model itself, so it excludes queueing, network and the API layer. Compare it with your own wall clock to see how much of the total the network takes. The OpenAI-compatible routes report the request's input tokens inside `usage` instead. Input tokens cost \$0.04 per million and output tokens cost \$0. See [Pricing](/reference/pricing).

## Backpressure

The scheduler answers `529 overloaded` when no slot frees in time. A failed call marks its slot down for 30 seconds and retries once on another slot. A second failure returns `502 runner_error`.

Both are transient. Retry with backoff. See [Errors](/reference/errors).

## Next

* [Long text and chunking](/concepts/long-text) — how chunking changes your results, and when to pre-split.
* [Decisions and probabilities](/concepts/decisions-and-probabilities) — the formulas behind every number.
* [Limits and rate limits](/reference/limits) — field limits and the per-organization rate limits.