OpenAI privacy-filter vs. GLiNER PII Comparison

When OpenAI privacy-filter, released under Apache 2.0, was evaluated with 600 ai4privacy PII samples, performance looked low under strict exact span matching, but the results changed significantly once a tokenizer artifact that shifted boundaries by 1 character was corrected.

• Dataset: 400 English samples, 200 samples each for French, German, Spanish, Italian, and Dutch
• Strict F1: GLiNER large-v2.1 0.367, privacy-filter 0.155
• Boundary F1: GLiNER 0.416, privacy-filter 0.498

English boundary F1 was as follows.

• EMAIL: privacy-filter 0.99, GLiNER 0.73
• PHONE: privacy-filter 0.67, GLiNER 0.51
• PERSON: privacy-filter 0.69, GLiNER 0.62
• DATE: privacy-filter 0.27, GLiNER 0.26
• ADDRESS: GLiNER 0.39, privacy-filter 0.37

Because of privacy-filter's GPT-style BPE tokenizer, token boundaries were misaligned with character boundaries, causing many correct detections to be counted as false positives under strict exact span matching. Conversely, under boundary scoring, EMAIL rose to 0.987 F1 for English and 1.000 for the multilingual set.

CPU processing speed also favored privacy-filter, at about 2.8 samples per second versus GLiNER's 1.1 samples/second, which the author attributed to its MoE sparse activation and 50M active parameters architecture. Notably, multilingual performance came out stronger than English performance.