Images | decision-machine-1 documentation

Every capability accepts an image field. You send the bytes as base64. The model reads the image first, then the same statement, labels, scale, question, schema, types or field you already send on text.

Send bytes, never a URL. The API never fetches an image for you. An http:// or https:// value returns 400.

The image field

image

string

One image, as a data:image/jpeg;base64,, data:image/png;base64, or data:image/webp;base64, URL, or as bare base64 of the same three formats. At most 5 MB decoded.

detail

'low' | 'medium' | 'high'Defaults to medium

The resolution the model reads. low is 512 px, medium is 768 px, high is 1024 px on the longest edge. The service does the resize.

text

string

Optional context that travels with the image. Send it when the decision needs facts the pixels do not carry.

One image per request. There is no images array, no multi-page input, no video. Rasterise a PDF page yourself and send the page as JPEG or PNG.

Send exactly one of text, texts or image. image may accompany text. image together with texts returns 400 with the code image_with_texts.

One call

IMAGE=$(base64 < receipt.jpg | tr -d '\n')
curl -X POST https://api.milliseconds.ai/v1/decision-machine-1/classify \
  -H "content-type: application/json" \
  -H "authorization: Bearer test_sk-..." \
  -d "{
    \"image\": \"data:image/jpeg;base64,$IMAGE\",
    \"detail\": \"low\",
    \"labels\": {
      \"invoice\": \"A bill that asks for payment\",
      \"receipt\": \"Proof that a payment happened\",
      \"contract\": \"An agreement between two parties\"
    }
  }"

Both SDKs take the same two fields. The TypeScript client accepts a Uint8Array, an ArrayBuffer, a Blob, bare base64 or a data URL, and imageFile(path) reads a file in Node. The Python client accepts bytes, a path, bare base64 or a data URL. detail takes low, medium or high in both. See TypeScript, Python and CLI (dm1).

Which capabilities read an image

Capability	On images	What the image changes
yes-no	Yes	One forward pass, as on text. Same `answer` and `probability`.
classify	Yes	Same `label`, `confidence` and `scores`.
classify-tree	Yes	One pass per level over the same image.
rate	Yes	Same ordered scale and `scores`.
answer	Yes	The model reads the pixels. Same `answer` and `probability`. An image has no character string, so `start` and `end` are `null`.
extract	Yes	Same schema, same `data`. The vision model fills arrays of objects too. At most 5 fields per schema.
entities	Yes	Same label-keyed items. `start`, `end` and `probability` are `null`.
verify	Yes	Same `matches` and `found`. `probability` is `1` or `0`, the verdict itself.

extract, entities and verify on an image are generative passes. The vision model writes the answer token by token. Use them for a handful of fields off one document image: a receipt total, an ID number, a date. Do not use them for long-form output or full-page transcription.

An image extract call takes 2.5 to 3.5 seconds. A decision on the same image takes about 90 ms. An image extract schema accepts at most 5 fields. A larger schema returns 400 with the code image_schema_too_large.

For a full page, parse or OCR the document to text first. Then send that text to the same capabilities. The text path is fast, and it sets no field cap. For one value off an image, use answer. For a verdict, use classify or yes-no.

question, questions, statement, statements, labels, tree, scale, schema, types, field and value keep their text meaning. You describe the decision in words, whatever the input is.

Billing

The base64 never enters the character count. You pay a fixed number of tokens for the image, plus the rest of the body counted as today.

billed input tokens = tokens(body without the image) + image tokens for the tier

`detail`	Longest edge	Image tokens	Model time (`yes-no`, `classify`, `classify-tree`, `rate`)
`low`	512 px	1,000	45 ms
`medium` (default)	768 px	2,000	90 ms
`high`	1024 px	4,000	120 ms

At $0.04 per million input tokens, a medium image costs $0.00008. Output stays free, so the full scores distribution costs nothing extra. See Pricing.

The generative capabilities run a longer pass on the same model, so they bill a multiple of the tier:

Capability	Multiple	`low`	`medium`	`high`
`yes-no`, `classify`, `classify-tree`, `rate`	x1	1,000	2,000	4,000
`answer`	x1.5	1,500	3,000	6,000
`extract`, `entities`, `verify`	x2	2,000	4,000	8,000

The OpenAI-compatible endpoint bills every image at the extract rate, x2, whatever the operation.

x-input-tokens reports the total, image tokens included. x-input-chars counts the body without the image. Token rate limits count the same tokens. The image itself adds no request to the request rate limit, but the batch rules are unchanged: each texts item, statement or question still counts as one request. See Limits and rate limits.

The image bills once for the request, whatever the number of statements, questions or tree levels. The tier tokens are added once to the total. On yes-no, classify, classify-tree and rate the extra items share the image on the GPU too, and cost about a tenth of the first. answer runs one generation per question, so its model time grows with the number of questions.

Size and format

5 MB decoded, per image. A larger image returns 400 with the code image_too_large.
JPEG, PNG and WebP only. Anything the service cannot decode returns 400 with the code invalid_image.
The check runs on the prefix and the length of your base64, before any decode. A bad image fails in the gateway, never on a GPU.
Resize before you send. A smaller upload transfers faster and bills the low tier. Test the tiers on your own pages: the accuracy below was measured at 768 px only.

Privacy

Images are processed in memory on the GPU machine. They are never written to disk and never logged. The inference log holds the task key and the top probability only. The zero-retention terms that cover your text cover your images.

What to expect

Measured on 100 items per set at 768 px:

Set	Result
16 document types, `classify`	61 % top-1
10 dish classes, photo `classify`	100 % top-1
”this document is a <type>”, `yes-no`	F1 0.82 at threshold 0.5, 0.88 at the best threshold
100 receipts, `extract`	fields 74 %, line items F1 0.82
100 document questions, `answer`	ANLS 0.90

These numbers come from one bake-off at 768 px on public sets, not from a guarantee. A document photograph is harder than an object photograph. Test your own pages before you pick a threshold. See Golden sets.

Input — the text fields and the batch envelope.
Extract — a typed record from a photograph or a scanned page.
Pricing — the token price and the free allowance.