OpenAI-compatible API

Streaming and differences

How stream:true behaves, and what the OpenAI surface does not implement.

The OpenAI surface is a compatibility shim over a decision model. It accepts the request shape your client already sends. It does not behave like a chat model.

Read this page before you point a production client at https://api.milliseconds.ai/v1.

Streaming

Set stream: true on POST /v1/chat/completions. The response has content-type: text/event-stream and cache-control: no-cache.

The server does the full decision first, then serializes it. The stream is always three data: lines:

  1. One chunk with the complete content or the complete tool call.
  2. One chunk with an empty delta and finish_reason, carrying usage.
  3. data: [DONE].

There is no incremental delta. No text arrives before the decision finishes.

Streaming does not reduce time to first token. The first chunk arrives after the model completes. Use it only for client compatibility.

Structured extraction stream

A real stream from the response_format.json_schema request below:

data: {"id":"chatcmpl-ab88beba37d64886aaaa6135","object":"chat.completion.chunk","created":1789607831,"model":"decision-machine-1","choices":[{"index":0,"delta":{"role":"assistant","content":"{\"restaurant\":\"Nobu\",\"party_size\":2}"},"finish_reason":null}]}
data: {"id":"chatcmpl-ab88beba37d64886aaaa6135","object":"chat.completion.chunk","created":1789607831,"model":"decision-machine-1","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":7,"completion_tokens":9,"total_tokens":16}}
data: [DONE]

Tool call stream

A real stream from tools, over the text Weather in Paris with a get_weather tool. Each tool call carries an index:

data: {"id":"chatcmpl-97c7c0978e44463a95976267","object":"chat.completion.chunk","created":1789606542,"model":"decision-machine-1","choices":[{"index":0,"delta":{"role":"assistant","tool_calls":[{"index":0,"id":"call_243386d795404cfa80ab6dcf","type":"function","function":{"name":"get_weather","arguments":"{\"city\":\"Paris\"}"}}]},"finish_reason":null}]}
data: {"id":"chatcmpl-97c7c0978e44463a95976267","object":"chat.completion.chunk","created":1789606542,"model":"decision-machine-1","choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}],"usage":{"prompt_tokens":4,"completion_tokens":4,"total_tokens":8}}
data: [DONE]

Call it

curl -N https://api.milliseconds.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "decision-machine-1",
    "stream": true,
    "messages": [{"role": "user", "content": "Book Nobu for 2 on Friday"}],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "booking",
        "schema": {
          "type": "object",
          "properties": {
            "restaurant": {"type": "string"},
            "party_size": {"type": "integer"}
          }
        }
      }
    }
  }'

No API key is required during the launch period. Pass any string, because the SDKs demand one.

Usage tokens

The usage object reports prompt_tokens, the request’s input tokens. It also reports completion_tokens and total_tokens.

The price is $0.04 per million input tokens and $0 per output token. See Pricing.

What the API ignores

The request body is permissive. Unknown and unsupported fields pass validation and change nothing.

FieldEffect
temperature, top_p, max_tokens, and similar sampling fieldsAccepted, ignored
system, developer, assistant, tool messagesDiscarded before inference
json_schema.name, json_schema.strictAccepted, ignored
required inside an extract schemaRead, never enforced

Only user messages carry text into the model. Multiple user turns join with blank lines.

A system prompt cannot steer a decision model. A pirate-speak system prompt changed nothing in testing.

What returns an error

RequestStatusCode
Plain chat, no response_format and no tools400unsupported_request
response_format: {"type":"text"} or {"type":"json_object"}400unsupported_request
tool_choice: "none" without a usable response_format400unsupported_request
No user message with text400invalid_request
tool_choice names a tool absent from tools400invalid_request
A model other than decision-machine-1404model_not_found
GET /v1/models/{id} with a wrong id404model_not_found

The plain-chat refusal is deliberate:

{"error":{"code":"unsupported_request","message":"decision-machine-1 decides, it does not chat. Send response_format.json_schema for structured extraction, tools for function calling, or use the capability endpoints under /v1/decision-machine-1/."}}

What does not exist

The OpenAI surface is three routes: GET /v1/models, GET /v1/models/{id} and POST /v1/chat/completions.

  • No embeddings. There is no /v1/embeddings endpoint.
  • No images. The facade keeps only the text parts of array content. Image parts never reach the model.
  • No audio, files, assistants or responses API. Those routes do not exist.
  • No multiple tool calls. One choice comes back, and it carries exactly one tool call.
  • No generation. Every output is a decision over the text you sent.

An unrouted path or a wrong method returns a plain-text 404 Not Found, outside the JSON error envelope.

Next