> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.milliseconds.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.milliseconds.ai/_mcp/server.

# Cascade to an LLM

Most production traffic is easy. A support ticket about a double charge is billing. A passage that never mentions the question does not answer it. A large model answers those cases correctly. It also answers them slowly.

A cascade puts `decision-machine-1` in front of the large model. Stage one decides. Stage two runs only when stage one is unsure.

## The shape

#### Call the capability

Send the text to the capability that matches the decision. Use [Classify](/capabilities/classify) for a label, [Yes / no](/capabilities/yes-no) for a flag, [Rate](/capabilities/rate) for a level.

#### Read the numbers

[Classify](/capabilities/classify) returns `probability` and `confidence` on the same response. `probability` is the winner's normalized share. `confidence` is `1 − normalized entropy` over `scores`.

#### Accept or escalate

Above your threshold, act on the decision and stop. Below it, call the large model with the text and the stage-one scores.

The large model never sees the easy cases. That is the whole saving.

## The gate

One gate decides everything. With `classify`, use both numbers.

| Stage-one result                               | Meaning                          | Action                                       |
| ---------------------------------------------- | -------------------------------- | -------------------------------------------- |
| `probability >= 0.90` and `confidence >= 0.70` | One clear winner                 | Act. No escalation.                          |
| `probability >= 0.90` and `confidence < 0.70`  | The winner leads a crowded field | Act on low blast radius. Escalate otherwise. |
| `probability < 0.90`                           | Two or more labels compete       | Escalate.                                    |

The three stage-one capabilities return different fields. Gate on the fields the capability actually returns.

| Capability                         | Fields returned                                   | Gate on                                                                          |
| ---------------------------------- | ------------------------------------------------- | -------------------------------------------------------------------------------- |
| [Classify](/capabilities/classify) | `label`, `probability`, `confidence`, `scores`    | Both numbers, as above.                                                          |
| [Rate](/capabilities/rate)         | `score`, `level`, `label`, `confidence`, `scores` | `confidence` and the largest entry in `scores`. `rate` returns no `probability`. |
| [Yes / no](/capabilities/yes-no)   | `statement`, `answer`, `probability`              | `probability` alone. `yes-no` returns no `confidence` and no `scores`.           |

On `yes-no`, `probability` is the share of the `when_true` hint when you send hints. Without hints it is one raw score. Escalate the middle band, for example `0.10 < probability < 0.90`.

These thresholds are a starting point, not a measurement. Tune them against a labelled set — see [Tuning thresholds](/evaluate/tuning-thresholds).

## Two real responses

This ticket is easy. Stage one is finished.

**`classify — act on this`**

```json title="classify — act on this"
{"label":"billing","probability":0.942,"confidence":0.797,"scores":{"billing":0.942,"shipping":0,"account":0.058}}
```

This rating is not. Two levels split the mass almost evenly, and `confidence` reports the split.

**`rate — escalate this`**

```json title="rate — escalate this"
{"score":1.585,"level":2,"label":"Frustrated","confidence":0.371,"scores":[0,0.471,0.472,0.057]}
```

> **Note**
>
> The second response is honest, not broken. `level` is 2 by a margin of 0.001. A cascade exists to catch exactly this case.

## Code

The gate is a few lines. Keep it in your own code, next to the action it guards.

#### curl

```bash
curl -s https://api.milliseconds.ai/v1/decision-machine-1/classify \
  -H 'content-type: application/json' \
  -H 'authorization: Bearer test_sk-...' \
  -d '{
    "text": "I was charged twice for my subscription this month and support has not replied.",
    "labels": {
      "billing": "payments, invoices, refunds, charges",
      "shipping": "delivery, tracking, returns in transit",
      "account": "login, password, profile settings"
    }
  }'
```

#### Python

```python
import requests

BASE = "https://api.milliseconds.ai/v1/decision-machine-1"
# ask_llm is your own stage-two call. It takes the text and the stage-one scores.
LABELS = {
    "billing": "payments, invoices, refunds, charges",
    "shipping": "delivery, tracking, returns in transit",
    "account": "login, password, profile settings",
}

def route(text):
    r = requests.post(f"{BASE}/classify", headers={"authorization": "Bearer test_sk-..."}, json={"text": text, "labels": LABELS})
    r.raise_for_status()
    d = r.json()
    if d["probability"] >= 0.90 and d["confidence"] >= 0.70:
        return {"label": d["label"], "source": "decision-machine-1"}
    return {"label": ask_llm(text, d["scores"]), "source": "llm"}
```

#### TypeScript

```typescript
const BASE = "https://api.milliseconds.ai/v1/decision-machine-1";
// askLlm is your own stage-two call. It takes the text and the stage-one scores.
const LABELS = {
  billing: "payments, invoices, refunds, charges",
  shipping: "delivery, tracking, returns in transit",
  account: "login, password, profile settings",
};

async function route(text: string) {
  const res = await fetch(`${BASE}/classify`, {
    method: "POST",
    headers: { "content-type": "application/json", authorization: "Bearer test_sk-..." },
    body: JSON.stringify({ text, labels: LABELS }),
  });
  const d = await res.json();
  if (d.probability >= 0.9 && d.confidence >= 0.7) {
    return { label: d.label, source: "decision-machine-1" };
  }
  return { label: await askLlm(text, d.scores), source: "llm" };
}
```

Pass `scores` to the large model. It names the labels that are in contention. `classify` and `rate` return `scores`; `yes-no` does not.

## The math

Call `e` the share of traffic that fails the gate. Stage one runs on every item. Stage two runs on `e` of them.

* **LLM calls:** `e` per item, down from 1 per item.
* **Mean latency:** `t1 + e × t2`, where `t1` is the measured stage-one latency below and `t2` is your large model's latency.
* **Stage-one cost:** \$0.04 per million input tokens and \$0 per output token. The cost scales with the length of the text and the labels you send. The `x-input-tokens` response header reports the input tokens billed for this call. See [Pricing](/reference/pricing).

At `e = 0.15`, 85 of every 100 items never reach the large model. The cascade still pays the fixed `t1` on all 100. Supply `t2` yourself: this page documents no large-model latency.

### Measured stage-one latency

Production, 2026-09-18, short inputs, wall clock from a laptop, median of 7 runs.

| Stage-one call          | Observed |
| ----------------------- | -------- |
| `classify`, 3 labels    | 0.43 s   |
| `yes-no`, one statement | 0.42 s   |
| `rate`, 4 levels        | 0.42 s   |
| `answer`, 1 question    | 0.40 s   |
| `entities`, 3 types     | 0.42 s   |

> **Warning**
>
> A cascade adds latency when `e` is high. A gate that escalates more than half of your traffic usually points at the label text, not at the model. Rewrite the labels with [Writing good statements and labels](/concepts/writing-good-questions) before you tune thresholds.

## Batch the first stage

Stage one takes up to 32 texts in one request. Screen a queue in one call, then escalate the failures individually.

**`Screen a batch, then escalate the failures`**

```bash title="Screen a batch, then escalate the failures"
curl -s https://api.milliseconds.ai/v1/decision-machine-1/yes-no \
  -H 'content-type: application/json' \
  -H 'authorization: Bearer test_sk-...' \
  -d '{
    "texts": ["First ticket text.", "Second ticket text."],
    "statement": "The customer expresses urgency.",
    "when_true": "The customer needs an answer today.",
    "when_false": "The customer can wait for a normal reply."
  }'
```

Results come back in `{"results":[...]}`, in input order, one entry per text. Each entry carries `statement`, `answer` and `probability`. Gate the batch on `probability`. Details in [Batching](/capabilities/batching).

> **Note**
>
> Every request needs an API key. See [Authentication](/authentication). Send the request as shown.

## Measure before you ship it

A cascade has one number that matters: the escalation rate `e`. Log it from day one, with the stage-one scores that produced it.

* `e` climbs over a week: your label set no longer covers the traffic.
* `e` falls to near zero: your gate is too loose, and wrong decisions pass it.
* The large model agrees with most escalations: your gate is too tight.

[Monitoring](/evaluate/monitoring) covers the distributions to watch.

## Next

* [Confidence routing](/patterns/confidence-routing) — the four-band version of this gate.
* [Tuning thresholds](/evaluate/tuning-thresholds) — turn a golden set into a threshold per action.
* [When to use the native API](/openai/when-to-use-native) — why stage one uses the capability endpoints, not the chat surface.