How it works
decision-machine-1 is not one large model. It is two small encoder models behind one API, plus a scheduler that keeps them busy.
Each call runs one pass over your text. It uses no sampling loop, no retry chain and no prompt parser. That is where the speed comes from.
The two models
The classifier scores labels. The extractor finds spans. Every capability maps to one of those two jobs.
Labels are the interface. The classifier reads your label text and scores it against the input. The words yes and no carry no meaning for it. See Writing good statements and labels.
What the classifier does
The classifier scores every label independently in the same pass, so adding labels adds their length to the input and nothing else.
The API then normalises those scores. Decisions and probabilities gives the exact formulas.
What the extractor does
The extractor finds the best span for each label you supply. extract sends your JSON Schema as a structure. answer sends each question as a label, plus one decoy label named other. The decoy sharpens the spans, and the API discards it. entities sends each type as a label. verify checks one field against one value.
The extractor is multilingual. The classifier performs best in English. See Languages.
One pass per call
Every capability call does the same three things:
The Worker builds the labels
A Cloudflare Worker turns your request into labels or structures. It validates the field limits first, so bad input fails in under 0.2 s.
Inference runs on CPU. The runner is a FastAPI service on two boxes, three processes per box. A lock holds each process to one inference at a time.
Chunking at 2,000 characters
Both models slow down past roughly 500 tokens. The pipeline splits any text over 2,000 characters on whitespace, so every chunk stays in the linear regime.
A single over-long token becomes its own chunk. extract also collapses runs of two or more spaces to " | " first, so table cells read as distinct spans.
Cost is linear in characters. Long text and chunking covers the practical guidance on pre-splitting.
Measured latency
Two sets of numbers. The first is the runner benchmark on a Xeon E-2136 (6c/12t), 3 slots x 2 threads, 1,000-character texts.
Hyperthreads add nothing. The box saturates near 4 extractor calls per second whatever the process and thread layout.
The second set is end-to-end wall clock, measured with curl from a laptop against production on 2026-09-16. Inputs were 55 to 198 characters. These numbers include TLS, internet transit and the Worker hop.
The split is clean: classifier calls land near 1 s, extractor calls near 0.5 s.
Batch the cheap axis. Extra statements or questions ride in one runner call and add only their own length to the billed input. Extra texts cost one call each. See Batching.
What you can measure yourself
Every capability response carries the input size in headers. Use them to predict cost and to size your batches.
x-input-tokens is the input tokens billed for this call. x-input-chars is the number of input characters, and it sums every item for texts. The OpenAI-compatible routes report the request’s input tokens inside usage instead. Input tokens cost $0.04 per million and output tokens cost $0. See Pricing.
Backpressure
The scheduler answers 529 overloaded when no slot frees in time. A failed call marks its slot down for 30 seconds and retries once on another slot. A second failure returns 502 runner_error.
Both are transient. Retry with backoff. See Errors.
Next
- Long text and chunking — how chunking changes your results, and when to pre-split.
- Decisions and probabilities — the formulas behind every number.
- Limits and rate limits — field limits and the launch-period terms.