Recipes

PII detection

Find personal data with described entity types, then redact it with the returned character offsets.

Support transcripts, form notes and uploaded documents carry personal data. A regular expression finds an email address. It does not find a name, a street address, or an account number written in prose.

The entities capability finds every span per type. Each span carries start and end, so your code can redact the text without a second search.

Describe the types

Send types as an object that maps a type name to a description. The description is what the model reads. You can also send an array of bare names, but each name then has to carry the whole definition.

curl -s https://api.milliseconds.ai/v1/decision-machine-1/entities \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Hi, this is Maria Gonzalez. You can reach me at maria.gonzalez@example.com or on 415-555-0142. My account number is 4471-8820 and I live at 88 Market Street, San Francisco.",
    "types": {
      "person_name": "the full name of a person",
      "email_address": "an email address",
      "phone_number": "a telephone number",
      "account_number": "a customer or bank account number",
      "street_address": "a postal or street address"
    }
  }'

The call above returns this response:

{
  "entities": [
    {"type":"person_name","text":"Maria Gonzalez","probability":1,"start":12,"end":26},
    {"type":"email_address","text":"maria.gonzalez@example.com","probability":0.998,"start":48,"end":74},
    {"type":"phone_number","text":"415-555-0142","probability":0.999,"start":81,"end":93},
    {"type":"account_number","text":"4471-8820","probability":0.999,"start":116,"end":125},
    {"type":"street_address","text":"88 Market Street, San Francisco","probability":0.999,"start":140,"end":171}
  ]
}

One broad name works less well. The same text with "types": ["pii"] returns two spans and misses three:

{
  "entities": [
    {"type":"pii","text":"maria.gonzalez@example.com","probability":0.952,"start":48,"end":74},
    {"type":"pii","text":"415-555-0142","probability":0.948,"start":81,"end":93}
  ]
}

One described type per kind of personal data finds more.

Redact with the offsets

The API sorts entities by start ascending. Walk the list in reverse and cut each span out of the original string. Reverse order keeps every remaining offset valid.

def redact(text, entities, floor=0.5):
    for e in sorted(entities, key=lambda e: e["start"], reverse=True):
        if e["probability"] < floor:
            continue
        text = text[: e["start"]] + f"[{e['type'].upper()}]" + text[e["end"] :]
    return text

The result:

Hi, this is [PERSON_NAME]. You can reach me at [EMAIL_ADDRESS] or on [PHONE_NUMBER]. My account number is [ACCOUNT_NUMBER] and I live at [STREET_ADDRESS].

Redaction destroys data. Use a low floor, near 0.5, and review the discarded spans. A missed span leaks; a wrong span only costs you a word.

Choose a probability floor

ProbabilityAction
0.9 and aboveRedact and release the text
0.5 to 0.9Redact, and keep the document in a review queue
Below 0.5Log the span, send the document to a person

Measure these bands against your own data before you trust them. Golden sets and threshold tuning give you the numbers.

Non-English text

The extractor behind entities is multilingual. The same English type descriptions work against French text. This input:

Bonjour, je suis Claire Dubois. Mon adresse est claire.dubois@exemple.fr et mon numero de telephone est le 06 12 34 56 78.

returns these spans:

{
  "entities": [
    {"type":"person_name","text":"Claire Dubois","probability":1,"start":17,"end":30},
    {"type":"email_address","text":"claire.dubois@exemple.fr","probability":0.996,"start":48,"end":72},
    {"type":"phone_number","text":"06 12 34 56 78","probability":1,"start":107,"end":121}
  ]
}

Write your type descriptions in English, whatever the language of the text. Every measured probe used English descriptions against non-English input.

Scale to a corpus

Send up to 32 documents per call in texts. The response is {"results": [...]}, one entry per document, in input order. The API runs a separate inference call per document, so a batch of 32 saves round trips, not work.

Documents over 2,000 characters run through overlapping windows. The API remaps the offsets back onto your original string. A single document holds up to 20,000 characters. Cost scales with the length of the text and the type descriptions you send: $0.04 per million input tokens and $0 per output token. See Pricing.

No API key is required during the launch period. A 529 response is the backpressure signal: retry it with backoff.

Next