Does replacing PHI with realistic surrogates stop downstream tools from finding it? Across 11 detectors and 7 benchmarks — no.
On the 57,112 spans Custodian masks, detector recall is 76.1% → 74.9% (−1.2 pts). An equivalence test (TOST, p≈3×10⁻⁹) confirms this is statistically equivalent to zero within ±2 points — too small to matter. Ranking is preserved; the flagship Llama 3.3-70B is unchanged.
Real values → realistic same-type surrogates. The sentence still reads normally, so anything that processed the original works on the result.
Recall on masked spans, transformed ÷ original (overlap match — does the detector find the surrogate?).
| System | Retention |
|---|
40,165 spans found in both; 3,300 lost (50% same length — not a boundary effect). The cause is surrogate quality, not detector failure:
All three are fixable surrogate-generation defects — not evidence that transformation hides well-formed PHI.
Detectability is preserved. Surrogates stay findable 93–100% of the time; ranking unchanged.
Coverage, reported separately. The transform masks ~80% of genuine clinical PHI (lower on general text); the utility result above is measured only on spans it actually masks.
▶ View the slides in your browser
A 5-slide walkthrough (EN / 中文 toggle) — no download needed.
📝 Read the workshop paper (PDF)
Draft for a clinical/privacy NLP workshop — the paired multi-detector protocol, TOST equivalence testing, and the redaction / open-surrogate comparison experiments.