> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.milliseconds.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.milliseconds.ai/_mcp/server.

# Content moderation

Moderation asks two different questions. *How bad is this?* is a scale. *Does this break a
named rule?* is a yes or no. Ask them separately and you get two numbers you can act on.

This recipe runs `rate` for severity and one batched `yes-no` call for the hard flags.
Every response below comes from a live call against `https://api.milliseconds.ai`.

| Question                        | Capability | What you act on                |
| ------------------------------- | ---------- | ------------------------------ |
| How severe is this post?        | `rate`     | `score`, `level`, `confidence` |
| Does it threaten, dox, or spam? | `yes-no`   | One `probability` per rule     |
| Who reviews the rest?           | Your code  | The confidence bands below     |

## Rate the severity

Write the scale as level descriptions, ordered low to high. A scale takes 2 to 10 levels.
Numbers ("1", "2", "3") carry no meaning for the model — describe each level.

#### curl

```bash
curl -X POST https://api.milliseconds.ai/v1/decision-machine-1/rate \
  -H 'content-type: application/json' \
  -H 'authorization: Bearer test_sk-...' \
  -d '{
    "text": "Honestly this is the laziest article I have read all year. Did anyone proofread it?",
    "scale": [
      "Civil and harmless",
      "Rude or dismissive",
      "Insulting or demeaning toward a person or group",
      "Hateful or threatening"
    ]
  }'
```

#### Python

```python
import requests

SCALE = [
    "Civil and harmless",
    "Rude or dismissive",
    "Insulting or demeaning toward a person or group",
    "Hateful or threatening",
]

def severity(post: str) -> dict:
    r = requests.post(
        "https://api.milliseconds.ai/v1/decision-machine-1/rate",
        headers={"authorization": "Bearer test_sk-..."},
        json={"text": post, "scale": SCALE},
        timeout=10,
    )
    r.raise_for_status()
    return r.json()
```

#### TypeScript

```typescript
const SCALE = [
  "Civil and harmless",
  "Rude or dismissive",
  "Insulting or demeaning toward a person or group",
  "Hateful or threatening",
];

async function severity(post: string) {
  const res = await fetch("https://api.milliseconds.ai/v1/decision-machine-1/rate", {
    method: "POST",
    headers: { "content-type": "application/json", authorization: "Bearer test_sk-..." },
    body: JSON.stringify({ text: post, scale: SCALE }),
  });
  return res.json();
}
```

```json
{"score":1.509,"level":1,"label":"Rude or dismissive","confidence":0.463,"scores":[0,0.501,0.488,0.011]}
```

Read all three numbers. `level` 1 is the argmax, but levels 1 and 2 split almost evenly
(0.501 against 0.488). `score` 1.509 sits between them, and `confidence` 0.463 reports the
ambiguity. This post belongs in a review queue, not in an automatic action.

> **Note**
>
> `score` is the probability-weighted position, `Σ p_i · i`. It ranges from 0 to
> `scale.length − 1`. `level` is the single most likely index. They disagree when the
> distribution spreads across levels, and that disagreement is your review signal.

## Flag the hard cases

A scale smears a rule violation across levels. Name each rule as its own statement and send
all of them in one `yes-no` call. The statements go into a single inference call, so the
batch costs about the same as one statement.

#### curl

```bash
curl -X POST https://api.milliseconds.ai/v1/decision-machine-1/yes-no \
  -H 'content-type: application/json' \
  -H 'authorization: Bearer test_sk-...' \
  -d '{
    "text": "You people are pathetic. I know where you live and I will make you regret posting this.",
    "statements": [
      "The post threatens violence against a person.",
      "The post attacks a person or group over race, religion, gender or origin.",
      "The post describes self-harm or suicidal intent.",
      "The post shares private personal information such as an address or phone number.",
      "The post is unsolicited advertising or spam."
    ]
  }'
```

#### Python

```python
import requests

RULES = [
    "The post threatens violence against a person.",
    "The post attacks a person or group over race, religion, gender or origin.",
    "The post describes self-harm or suicidal intent.",
    "The post shares private personal information such as an address or phone number.",
    "The post is unsolicited advertising or spam.",
]

def flags(post: str) -> dict:
    r = requests.post(
        "https://api.milliseconds.ai/v1/decision-machine-1/yes-no",
        headers={"authorization": "Bearer test_sk-..."},
        json={"text": post, "statements": RULES},
        timeout=10,
    )
    r.raise_for_status()
    return {x["statement"]: x["probability"] for x in r.json()["results"]}
```

#### TypeScript

```typescript
const RULES = [
  "The post threatens violence against a person.",
  "The post attacks a person or group over race, religion, gender or origin.",
  "The post describes self-harm or suicidal intent.",
  "The post shares private personal information such as an address or phone number.",
  "The post is unsolicited advertising or spam.",
];

async function flags(post: string) {
  const res = await fetch("https://api.milliseconds.ai/v1/decision-machine-1/yes-no", {
    method: "POST",
    headers: { "content-type": "application/json", authorization: "Bearer test_sk-..." },
    body: JSON.stringify({ text: post, statements: RULES }),
  });
  const body = await res.json();
  return Object.fromEntries(body.results.map((r) => [r.statement, r.probability]));
}
```

```json
{"results":[
  {"statement":"The post threatens violence against a person.","answer":true,"probability":1},
  {"statement":"The post attacks a person or group over race, religion, gender or origin.","answer":false,"probability":0.122},
  {"statement":"The post describes self-harm or suicidal intent.","answer":false,"probability":0.154},
  {"statement":"The post shares private personal information such as an address or phone number.","answer":false,"probability":0.243},
  {"statement":"The post is unsolicited advertising or spam.","answer":false,"probability":0.3}
]}
```

Each probability stands on its own. The five numbers do not sum to 1, so read every
flag against its own threshold. Here the doxxing flag reaches 0.243 on "I know where you
live" and still stays below any action band.

The same post scored `{"score":2,"level":3,"label":"Hateful or threatening","confidence":0.208,"scores":[0,0.333,0.333,0.333]}`
on the severity scale — a three-way tie. The scale was unsure; the threat statement was not.
This is why you run both calls.

The rude article comment returns the opposite picture. Its highest flag is spam at
probability 0.141, and the threat flag drops to 0.008.

> **Warning**
>
> Write each statement as a case, never as a verdict. Hint text such as `when_true: "yes"` and
> `when_false: "no"` carries no meaning to the model. A measured probe flipped a correct
> `answer: true` (probability 1) to `answer: false` (probability 0.004) with that wording.

## Route the item

Set one threshold per action, not one for the system. Removal needs more evidence than a
warning label does. Raise the bar with the blast radius.

| Signal                                                   | Band   | Action                       |
| -------------------------------------------------------- | ------ | ---------------------------- |
| Self-harm flag above 0.5                                 | Act    | Route to the crisis workflow |
| Any other hard flag above 0.9                            | Act    | Remove and notify            |
| `score` at or above 2.0 and `confidence` at or above 0.7 | Act    | Remove                       |
| `score` at or above 1.0 with `confidence` below 0.5      | Review | Human queue                  |
| `score` below 1.0 and every flag below 0.3               | Act    | Publish                      |

Build the `flags` dict from the `yes-no` response, then route on both calls.

```python
SELF_HARM = "The post describes self-harm or suicidal intent."

# flags = {r["statement"]: r["probability"] for r in body["results"]}

def route(rating: dict, flags: dict[str, float]) -> str:
    if flags[SELF_HARM] >= 0.5:
        return "crisis"
    if max(flags.values()) >= 0.9:
        return "remove"
    if rating["score"] >= 2.0 and rating["confidence"] >= 0.7:
        return "remove"
    if rating["score"] < 1.0 and max(flags.values()) < 0.3:
        return "publish"
    return "review"
```

Tune these numbers against your own labelled examples. The bands above are a starting shape,
not a measured policy.

## Batch a queue

Swap `text` for `texts` and keep the same `scale`. Send up to 32 posts. The results come
back in input order, one entry per post — a compliment, the rude comment, and the threat:

```json
{"results":[
  {"score":0,"level":0,"label":"Civil and harmless","confidence":1,"scores":[1,0,0,0]},
  {"score":1.509,"level":1,"label":"Rude or dismissive","confidence":0.463,"scores":[0,0.501,0.488,0.011]},
  {"score":2,"level":3,"label":"Hateful or threatening","confidence":0.208,"scores":[0,0.333,0.333,0.333]}
]}
```

The clean post returns `confidence` 1 and `score` 0. Publish it without a human.

## Operational notes

* Each capability takes at most 32 `texts`, and `yes-no` at most 32 `statements`.
* Batching `texts` costs one inference call per text. Batching `statements` does not.
* Measured latency: `rate` with 4 levels 0.42 s, `yes-no` with 2 statements 0.49 s.
* A `text` holds up to 20,000 characters. The API chunks text over 2,000 characters, and
  each level keeps its highest score over the chunks. A long thread scores on its worst part.
* Write scale and statement text in English, even for non-English posts.
* Handle `overloaded` (529) with backoff. Decide in advance whether a failed check holds
  the post or publishes it.

> **Note**
>
> Every request needs an API key. See [Authentication](/authentication).

## Next

* [Thresholds and confidence routing](/concepts/thresholds) turns these bands into policy.
* [Rate](/capabilities/rate) documents `score`, `level` and `scores` in full.
* [LLM guardrails](/patterns/llm-guardrails) applies the same flags to model input and output.