How to run PII detection on long documents

For experiment 007 we ran seven PII models over 7,820 long legal and medical letters and 6,000 shorter texts in six languages. Most of the work was not picking a model but feeding it the text the right way. This is what that takes.

Split the text first

GLiNER-based PII models, NVIDIA's gliner-PII and our own among them, read a fixed number of tokens per call: 384 for most PII checkpoints, which is about 300 words of ordinary prose. The GLiNER library cuts anything longer and prints one warning:

UserWarning: Sentence of length 612 has been truncated to 384

The letters in SPY, a public set of synthetic legal questions and medical consultations, are 419 words long at the median, and 98% of them run past 384 tokens. We ran four GLiNER models on the whole letters and on the same letters split into 200-word windows. A name or number counts as masked when every letter and digit of it falls inside something the model flagged.

max_len is a field in each checkpoint's gliner_config.json, so you can check yours without loading the model:

import json
from huggingface_hub import hf_hub_download

config = json.load(open(hf_hub_download("nvidia/gliner-PII", "gliner_config.json")))
print(config["max_len"])  # 384

Knowledgator's checkpoint reads 2,048 tokens, and Fastino's GLiNER2-PII runs on a library that doesn't cut. Both still did slightly better in windows.

Windows that overlap, and own their middle

We use 200-word windows that overlap by 40 words. A name that sits on the edge of one window is whole inside the next. To keep it from being found twice, each window keeps only the detections that start in the part of the text it owns, which runs from the middle of one overlap to the middle of the next.

import re

def windows(text, size=200, overlap=40):
    """Overlapping windows of `size` words as (start, end, own_start, own_end)
    character offsets. Each window keeps only what starts in the part it owns,
    so every character belongs to exactly one window."""
    words = [m.start() for m in re.finditer(r"\S+", text)]
    if len(words) <= size:
        return [(0, len(text), 0, len(text))]
    stride = size - overlap
    starts = list(range(0, len(words) - overlap, stride))
    out = []
    for k, first in enumerate(starts):
        last = min(first + size, len(words))
        end = words[last] if last < len(words) else len(text)
        own_start = 0 if k == 0 else words[first + overlap // 2]
        own_end = len(text) if k == len(starts) - 1 else words[starts[k + 1] + overlap // 2]
        out.append((words[first], end, own_start, own_end))
    return out

def predict_long(model, text, labels, threshold=0.5):
    found = []
    for start, end, own_start, own_end in windows(text):
        for e in model.predict_entities(text[start:end], labels, threshold=threshold):
            if own_start <= e["start"] + start < own_end:
                found.append({**e, "start": e["start"] + start, "end": e["end"] + start})
    return found

We checked it against all 8,688 SPY letters: every character belongs to exactly one window, and no labelled name or number is cut by the edge of the window that owns it.

Ask only for the labels you need

A GLiNER model reads the list of labels together with the text, so every label you ask for makes each call longer. Same container, same 20 letters, two CPU cores, asking for more and more of each model's own labels:

From five labels to all of them, each model gets about a third slower. If you only mask names, emails and phone numbers, ask for those.

Let patterns catch what has a pattern

Emails, phone numbers, IP addresses and ID numbers with a check digit have a shape that a regular expression finds reliably, and faster than any model. In the English part of ai4privacy, none of the GLiNER models flagged any of the 147 IPv6 addresses. pii-proxy's regex layer masks 94 to 99% of the IP addresses in each of six languages. With the regex layer behind NVIDIA's model, the share of personal data masked in English went from 87.9% to 91.5%.

What a million letters cost

On one A10G GPU, batches of 16, the windows of a ~450-word letter take:

On two CPU cores with 4 GB, one letter at a time and every label asked for, pii-proxy-ner takes 1.3 s a letter, about $47 per million at Modal's CPU and memory price, and NVIDIA's takes 7.1 s, about $248. For volume, a GPU is both faster and cheaper.

Languages without spaces

Thai is written without spaces between words, and GLiNER models can only start and end a span at their own word boundaries. Put a word segmenter in front. With PyThaiNLP's newmm inserting spaces between Thai words, GLiNER2-PII masked 91% of the Thai names in a synthetic Thai test set we wrote, against 50% without it.

What these numbers don't cover

SPY is written by an LLM. Real documents are longer or shorter, and put their personal data in other places.

200 words with a 40-word overlap is the setting we used, not one we tuned.

The timings are PyTorch with 32-bit weights; quantized ONNX would be faster and was not measured. GPU times exclude loading the model. Prices are Modal's list prices on 15 October 2026, GPU time only, without the container's CPU and memory.

Every number, the window code as it ran, and the scorer are in experiment 007.