Composite scoring

A broad question like “how serious is this ticket?” hides several judgments. The model answers it with one number that you cannot explain, audit or tune.

Composite scoring splits the question. You send one rate call per dimension, each with its own scale. You keep the weights in your code, where you can read them, test them and change them without touching the model.

Design the dimensions

Each dimension is one rate call with one ordered scale. Follow three rules.

• Keep the dimensions independent. Tone, business impact and time pressure vary on their own. Do not add a dimension that restates another one.
• Describe every level. The level text is what the model reads. "Low" and "High" carry no meaning. Write "The whole business is blocked" instead.
• Keep scales short. A scale holds 2 to 10 levels. Four clear levels beat ten vague ones.

More guidance on level wording lives in writing good statements and labels.

Score each dimension

The example text is one inbound support ticket:

We have been down since 09:00. Your API returns 500 on every checkout call. This is our busiest
trading day and we are losing orders. If this is not fixed today we will move to another provider.

Three calls run against it, one per dimension. The curl call below scores the impact dimension.

curl -s https://api.milliseconds.ai/v1/decision-machine-1/rate \
  -H "content-type: application/json" \
  -d '{
    "text": "We have been down since 09:00. Your API returns 500 on every checkout call. This is our busiest trading day and we are losing orders. If this is not fixed today we will move to another provider.",
    "scale": ["No work is blocked", "One person is blocked", "A team is blocked", "The whole business is blocked"]
  }'

The three responses, captured live:

{"score":2,"level":2,"confidence":0.208,"scores":[0,0.333,0.333,0.333]}

Use score, not level. score is the probability-weighted position Σ p_i · i, so it carries the model’s uncertainty. level is the argmax alone, and it can jump on a 0.001 difference.

Combine the scores

Normalise each score to 0–1 by dividing it by scale.length - 1. Then take the weighted sum.

def composite(results):
    parts = [w * results[n]["score"] / (len(s) - 1)
             for n, (w, s) in DIMENSIONS.items()]
    return sum(parts) / sum(w for w, _ in DIMENSIONS.values())
def triage(results):
    value = composite(results)
    floor = min(r["confidence"] for r in results.values())
    if floor < 0.35:
        return "review", value
    if value >= 0.75:
        return "page on-call", value
    if value >= 0.45:
        return "queue today", value
    return "normal queue", value

The captured numbers normalise to 0.667, 0.836 and 0.833, and give a composite of 0.793. That clears the paging band. The tone call reports confidence 0.208, so this ticket still routes to a person first. The model separated impact and urgency cleanly, and stayed unsure about tone.

Latency and batching

Each dimension is one call. A rate call runs near 1.0 s, so three dimensions in series cost near 3.0 s. Run them together with Promise.all or a thread pool. Separate inference slots take the three calls, so they finish in little more than the time of one.

Batching does not merge dimensions. texts sends many texts through one scale, so it scores many tickets on one dimension, not one ticket on many dimensions. To score a queue, batch up to 32 texts inside each dimension and read the results in input order, as batching describes.

Next

• Rate — the full request and response shape for one dimension.
• Confidence routing — the act, confirm and escalate bands used above.
• Tuning thresholds — pick the weights and the cut-offs from data.