Intent routing

A router decides where a message goes. A prompt-based router returns prose that you parse, and it drifts when you edit the wording. Classify returns one label name, a probability for that label, a score for every route, and a confidence number. One call takes about one second.

Use this pattern in front of a tool set, an agent hand-off, or a work queue. Use the OpenAI-compatible surface instead when your caller already speaks the OpenAI chat API.

The shape

Route a support message

The object form of labels maps each route name to its description. The model reads "name: description", so both parts carry weight.

curl -X POST https://api.milliseconds.ai/v1/decision-machine-1/classify \
  -H "content-type: application/json" \
  -d '{
    "text": "My card was declined twice but the money left my account. Can you check?",
    "labels": {
      "billing_issue": "A problem with a payment, a charge, a refund or an invoice",
      "track_order": "The customer asks where a shipment is or when it arrives",
      "cancel_subscription": "The customer wants to end a plan or a recurring subscription",
      "technical_support": "The product does not work: errors, login failures, broken features",
      "other": "Anything that no other label describes"
    }
  }'

The response, captured live in 1.15 s:

{
  "label": "billing_issue",
  "probability": 0.649,
  "confidence": 0.522,
  "scores": {
    "billing_issue": 0.649,
    "track_order": 0.001,
    "cancel_subscription": 0.025,
    "technical_support": 0.005,
    "other": 0.321
  }
}

scores sums to 1 over the label names. The runner-up here is other at 0.321, and confidence 0.522 reports that spread. The winner is right, but the router is not certain.

Dispatch on the numbers

Pick one band per route, not one band for the router. A read-only lookup can act at 0.5. A refund or a cancellation should ask first. Thresholds and confidence routing covers the method.

HANDLERS = {
    "billing_issue": open_billing_case,
    "track_order": lookup_shipment,
    "cancel_subscription": start_cancellation,
    "technical_support": open_support_ticket,
}
# Example bars. Raise each one with the blast radius of the action.
ACT_AT = {
    "track_order": 0.50,
    "technical_support": 0.60,
    "billing_issue": 0.70,
    "cancel_subscription": 0.85,
}
MIN_CONFIDENCE = 0.40  # a flat distribution means no winner
def dispatch(text):
    d = route(text)
    label, p = d["label"], d["probability"]
    if label == "other" or p < ACT_AT.get(label, 0.75):
        return ask_the_user(text, d["scores"])
    if d["confidence"] < MIN_CONFIDENCE:
        return ask_the_user(text, d["scores"])
    return HANDLERS[label](text)

The numbers above are examples, not product defaults. Measure your own bars against a labelled set. Tuning thresholds turns that set into a number per route.

Route over the OpenAI surface

Send your existing tools array to https://api.milliseconds.ai/v1. The API classifies the text over the tool descriptions, picks one tool, then fills its parameters schema with extract. Exactly one tool call comes back, with finish_reason: "tool_calls".

from openai import OpenAI
client = OpenAI(base_url="https://api.milliseconds.ai/v1", api_key="not-required")
completion = client.chat.completions.create(
    model="decision-machine-1",
    messages=[{"role": "user", "content": "My card was declined twice but the money left my account. Can you check?"}],
    tools=[
        {"type": "function", "function": {
            "name": "refund_or_charge_issue",
            "description": "A problem with a payment, a charge, a refund or an invoice",
            "parameters": {"type": "object", "properties": {
                "issue": {"type": "string", "description": "What went wrong with the payment"}}},
        }},
        {"type": "function", "function": {
            "name": "track_order",
            "description": "The customer asks where a shipment is or when it arrives",
            "parameters": {"type": "object", "properties": {
                "order_id": {"type": "string", "description": "The order number"}}},
        }},
    ],
)
print(completion.choices[0].message.tool_calls[0].function)

The captured reply, 1.33 s:

{
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_237bd20bedc54392b1a0b9b2",
            "type": "function",
            "function": {
              "name": "refund_or_charge_issue",
              "arguments": "{\"issue\":null}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ],
  "usage": {"prompt_tokens": 18, "completion_tokens": 4, "total_tokens": 22}
}

A field the text does not support comes back as null. Treat a null argument as a missing slot and ask for it.

Four rules govern this path. The API uses a single tool in the array directly, with no classify call. A classify winner that names no tool in the array falls back to the first tool. tool_choice: {"type":"function","function":{"name":"..."}} forces that tool and skips the routing step. tool_choice: "none" turns the tools path off, so the request needs response_format.json_schema or it returns 400 unsupported_request.

Native or OpenAI

The OpenAI surface returns no probability and no scores, so you cannot set a confidence band on it. When to use the native API lists the full trade.

Route a backlog

texts classifies up to 32 messages in one request. The response is {"results":[...]} in input order, so you can zip it against your queue. Each text costs one inference call. The calls spread across the inference slots, so a large batch takes longer than a single message. Batching covers the ordering and the limits.

Next

• Classify documents every field this pattern reads.
• Thresholds and confidence routing sets the per-route bars.
• Taxonomy classification walks a deep route tree level by level.