Long text and chunking

A single call accepts up to 20,000 characters in text. Past 2,000 characters, the API splits the text and runs the model once per piece. The split is internal. You always get one result per input text.

Both models slow down past roughly 500 tokens. The classifier backs yes-no, classify and rate. The extractor backs answer, extract, entities and verify. The 2,000-character chunk keeps every call in the linear regime, so cost stays linear in the character count.

Every capability response reports the input size in two headers. x-input-chars is the number of input characters. x-input-tokens is the input tokens billed for this call.

How the split works

The splitter cuts on whitespace. It never cuts inside a word. A single over-long token becomes its own chunk.

Each capability then merges the per-chunk results a different way.

The max rule protects recall. A claim that holds only in the last paragraph still scores high on its own chunk, so the chunk boundary does not hide it.

Offsets stay valid

answer, entities and verify return start and end offsets into the text you sent. For entities and verify, the API remaps the window offsets back to that text. You can slice the original string directly.

This call sent 2,469 characters. The last entity sits well past the 2,000-character boundary:

{"entities":[
  {"type":"place","text":"northern corridor","probability":0.984,"start":112,"end":129},
  {"type":"person","text":"Amelia Okafor","probability":0.995,"start":2432,"end":2445},
  {"type":"place","text":"Rotterdam depot","probability":0.967,"start":2453,"end":2468}
]}

text.slice(2432, 2445) on the original input returns Amelia Okafor.

Long text dilutes a classification

Chunking keeps a late signal readable. It does not isolate it. A short decisive sentence inside a long off-topic document shares its chunk with everything around it.

These three calls used the same three labels. The only difference is how much text went in.

The billing request scored 1 on its own. Inside 2,889 characters of logistics prose it never won a chunk. Split the text yourself when you need per-part answers.

Pre-split and batch

Send the parts as texts. The API returns {"results":[...]} in the same order. The batch limit is 32 texts per call. Each text is a separate inference call, so a batch of 8 runs 8 calls.

curl -s -X POST https://api.milliseconds.ai/v1/decision-machine-1/classify \
  -H 'content-type: application/json' \
  -d '{
    "texts": [
      "Dock scheduling stayed inside the agreed window. No carrier missed a slot.",
      "The customer asks us to cancel the annual subscription and refund the last invoice."
    ],
    "labels": {
      "billing": "the message is about invoices, payments, refunds or subscription charges",
      "logistics": "the message is about warehouses, shipping, carriers or packaging"
    }
  }'

Practical guidance

• Split on the structure you already have. Paragraphs, email messages, ticket replies, table rows and retrieved passages are natural units. Keep each unit under 2,000 characters.
• Keep a field’s context together. The merge rule takes the first non-empty value per field, so keep a label and its value in one chunk. Never cut between Total due: and the amount.
• Send the whole document for entities. Offsets stay correct, and you want every mention.
• Send the whole document for a document-level verdict. The max rule finds a late claim.
• Split when you need per-part verdicts. One call decides once per text.
• Cut boilerplate first. Signatures, legal footers and quoted history add characters and cost without adding signal.
• Watch the limits. text accepts 20,000 characters and texts accepts 32 items. Over either limit the API returns invalid_request with status 400.

Next