> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.milliseconds.ai/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.milliseconds.ai/_mcp/server. # Thresholds and confidence routing A threshold is a property of the action, not of the model. The same `classify` call can auto-file a ticket and block a refund. One of those tolerates mistakes. The other does not. Set one number per action. Then route every decision into three bands: act, confirm, escalate. ## The two numbers you route on | Field | What it measures | Where it appears | | ------------- | ---------------------------------------------------- | --------------------------------- | | `probability` | How much of the mass sits on the returned outcome | Every capability except `extract` | | `confidence` | `1 − normalized entropy` over the score distribution | `classify` and `rate` | `probability` answers "how sure about this one outcome". `confidence` answers "how separated were the alternatives". A winner at 0.51 against one rival differs from a winner at 0.51 against nine. Read [Decisions and probabilities](/concepts/decisions-and-probabilities) for what each field does and does not mean. > **Note** > > **Use both axes on `classify` and `rate`.** A low `confidence` means the scores stayed flat across the > labels. Send that case to a person, not to an action. ## Three bands | Band | Rule | What your code does | | -------- | ----------------------------- | ------------------------------------------------------------- | | Act | `probability >= act` | Apply the decision without a human | | Confirm | `review <= probability < act` | Apply it, but show a person the decision and let them undo it | | Escalate | `probability < review` | Do not act. Queue the case, or send it to a larger model | Two numbers per action define the bands. Start with a wide confirm band. Narrow it once a golden set tells you what you lose. `rate` returns no `probability` field. Route `rate` calls on `score` and `confidence` instead. ## Raise the bar with the blast radius Rank each action by what a wrong decision costs. Then set the `act` number against that cost. | Action | Cost of a mistake | Starting `act` | | ---------------------------------------------- | ----------------------------- | --------------- | | Add a tag, pick a queue, drop a RAG passage | A second of human time | 0.60 | | Set a priority, route to a team, redact a span | A misrouted case | 0.75 | | Auto-reply, close a ticket, publish a record | A visible error to a customer | 0.90 | | Block a user, issue a refund, delete data | Money or trust lost | Do not auto-act | > **Warning** > > These starting points are not measurements. They are a place to begin before you have data. > Replace every one of them with a number from your own golden set. > [Tuning thresholds](/evaluate/tuning-thresholds) turns that set into numbers. ## A worked pair Both responses below are real captures. The first is a clean win. The second is a near tie. **`classify — act`** ```json title="classify — act" {"label":"billing","probability":0.942,"confidence":0.797,"scores":{"billing":0.942,"shipping":0,"account":0.058}} ``` **`rate — escalate`** ```json title="rate — escalate" {"score":1.585,"level":2,"label":"Frustrated","confidence":0.371,"scores":[0,0.471,0.472,0.057]} ``` The `rate` call returns `level` 2, but level 1 scored 0.471 against level 2 at 0.472. The `confidence` of 0.371 reports that tie, and `score` 1.585 sits between the two levels. A threshold on `level` alone would hide this. A threshold on `confidence` catches it. > **Tip** > > On `rate`, route on `score` and `confidence` rather than on `level`. `score` is the probability-weighted > position, so it moves smoothly. `level` is an argmax and can flip on 0.001. ## Route in code Make the call, then apply your own bands to the response. **`curl`** ```bash title="curl" curl -s -X POST https://api.milliseconds.ai/v1/decision-machine-1/classify \ -H "Content-Type: application/json" \ -H "authorization: Bearer test_sk-..." \ -d '{ "text": "I was charged twice for my subscription this month and support has not replied.", "labels": { "billing": "a payment, invoice, refund or charge problem", "shipping": "a delivery, tracking or address problem", "account": "a login, password or profile problem" } }' ``` **`Python`** ```python title="Python" import requests ACT, REVIEW, MIN_CONFIDENCE = 0.85, 0.60, 0.50 text = "I was charged twice for my subscription this month and support has not replied." r = requests.post( "https://api.milliseconds.ai/v1/decision-machine-1/classify", headers={"authorization": "Bearer test_sk-..."}, json={ "text": text, "labels": { "billing": "a payment, invoice, refund or charge problem", "shipping": "a delivery, tracking or address problem", "account": "a login, password or profile problem", }, }, timeout=30, ) d = r.json() if d["probability"] >= ACT and d["confidence"] >= MIN_CONFIDENCE: band = "act" elif d["probability"] >= REVIEW: band = "confirm" else: band = "escalate" ``` **`TypeScript`** ```typescript title="TypeScript" const ACT = 0.85, REVIEW = 0.60, MIN_CONFIDENCE = 0.5; const text = "I was charged twice for my subscription this month and support has not replied."; const res = await fetch( "https://api.milliseconds.ai/v1/decision-machine-1/classify", { method: "POST", headers: { "Content-Type": "application/json", authorization: "Bearer test_sk-..." }, body: JSON.stringify({ text, labels: { billing: "a payment, invoice, refund or charge problem", shipping: "a delivery, tracking or address problem", account: "a login, password or profile problem", }, }), }, ); const d = await res.json(); const band = d.probability >= ACT && d.confidence >= MIN_CONFIDENCE ? "act" : d.probability >= REVIEW ? "confirm" : "escalate"; ``` Every request needs an API key. See [Authentication](/authentication). ## Do not inherit the built-in 0.5 Two fields already carry a fixed cut-off. `yes-no` sets `answer` to `true` when `probability >= 0.5`. `verify` sets `matches` the same way. That boundary serves the response shape, not your action. Read `probability` and apply your own number. A `true` at 0.51 and a `true` at 0.99 are different cases. ## Per-capability notes #### yes-no Without hints, `probability` is one raw sigmoid score and stays unnormalized. With `when_true` or `when_false`, it becomes `p_true / (p_true + p_false)`. The two forms produce different distributions, so tune one threshold per form. Hint text must describe the case, never the verdict. #### classify and rate `scores` is normalized and sums to 1. On `classify` it maps label name to probability. On `rate` it is one number per level, in `scale` order. Gate `classify` on `probability` and `confidence` together. With many labels, compare the top two scores before you act. #### answer and entities `probability` is the span confidence from the extractor. `answer` returns `null` with `probability` 0 when nothing fits, which is a clean escalate signal on its own. #### extract and verify `extract` returns no per-field probability. Check a field you care about with `verify`, then route on that `probability` and on `found[]`. The [Extract, then verify](/patterns/extract-then-verify) pattern shows the full loop. ## Next #### [Confidence routing](/patterns/confidence-routing) The full pattern, with act, confirm, review and escalate in Python and TypeScript. #### [Tuning thresholds](/evaluate/tuning-thresholds) Turn a golden set into precision and recall at each cut-off, then pick numbers per action. #### [Cascade to an LLM](/patterns/cascade) Send only the escalate band to a large model, and keep the cost of the rest flat.