Everything needed to re-run the structure-preservation study: the scoring scripts, the 250-document benchmark subsets (original + transformed), and the workshop paper. Public and self-contained — no proprietary access required. 复现这次结构保持性评测所需的一切：评分脚本、250 文档 benchmark 子集（原始 + 脱敏后）、以及 workshop 论文。公开、自包含 —— 无需任何私有权限。
📝 Workshop paper (PDF)Workshop 论文（PDF）
The paired multi-detector protocol, TOST equivalence testing, and the C1/C2/C3 comparison experiments. Every table below is produced by the scripts in §02.配对多检测器协议、TOST 等价检验、C1/C2/C3 对比实验。论文里的每张表都由 §02 的脚本产出。
***** and re-detect. Produces Table 3.C1 — 遮蔽地板值：把被遮蔽值换成 ***** 再检测。产出表 3。transform to sample text and checks the surrogate stays detectable, end-to-end. Needs a Custodian API key for the transform step; the evaluation runs free. (download .ipynb)一键运行的 notebook —— 对样例文本调用 Guardian Layer transform，再验证替身仍可被检测，端到端跑通。transform 步骤需 Custodian API key；评测部分免密钥。（下载 .ipynb）Each benchmark: the original subset and the Guardian-Layer-transformed subset, aligned by doc_id with gold PHI spans.每个 benchmark：原始子集与 Guardian Layer 脱敏后子集，按 doc_id 对齐、带 gold PHI span。
| Benchmark基准 | Original | Transformed |
|---|---|---|
| ASQ-PHI (en, clinical) [cite] | .jsonl | .jsonl |
| MEDDOCAN (es, clinical) | .jsonl | .jsonl |
| MultiCoNER v2 (multi) | .jsonl | .jsonl |
| PII-300k (en) | .jsonl | .jsonl |
| PII-300k (nl) | .jsonl | .jsonl |
| PII-300k (fr) | .jsonl | .jsonl |
| PII-300k (de) | .jsonl | .jsonl |
# deps: numpy scipy scikit-learn presidio-analyzer faker + spaCy models
pip install presidio-analyzer presidio-anonymizer faker
# C3 — equivalence (pooled + per-benchmark, McNemar, TOST Δ∈{1,2,3})
python scripts/analyze_equivalence.py
# Build the C1 (redact) + C2 (faker) corpora for all 7 benchmarks
PYTHONPATH=. python scripts/build_c1c2_corpora.py
# C1/C2 — Presidio baseline (CPU)
PYTHONPATH=. python scripts/run_redact_baseline.py
PYTHONPATH=. python scripts/run_faker_baseline.py
# C1/C2 — full panel across detector families (OBI CPU; Qwen/Gemma GPU)
# 120-doc/benchmark subsample for all three panel detectors
PYTHONPATH=. python scripts/score_corpus.py --system obi --all --limit 120
PYTHONPATH=. python scripts/score_corpus.py --system qwen3_5_9b --all --limit 120
PYTHONPATH=. python scripts/score_corpus.py --system gemma_4_31b --all --limit 120
# Assemble Tables 3 (C1) / 4 (C2), with 95% bootstrap CIs
python scripts/aggregate_c1c2.py --ci
# §7 surrogate-quality typology: AI review of 200 spans (human double-annotation optional)
python scripts/prefill_annotation.py # -> data/annotation/*_prefilled.csv
# review in the browser: open web/annotate.html (offline, keyboard-driven)
python scripts/annotation_summary.py r1.csv r2.csv # valid-rate + Cohen's kappa
Scripts read the paired result files under results/. Presidio runs on the full corpus (CPU); OBI (CPU) and the open-LLM cells (Qwen-9B, Gemma-31B, GPU) use a 120-doc/benchmark subsample. The equivalence analysis is pure re-analysis (no model inference). The §7 surrogate-quality typology is quantified by an AI review of 200 masked spans (not human-validated; labels in data/annotation/surrogate_quality_audit.csv); an offline tool annotate.html supports optional human double-annotation (guidelines in data/annotation/GUIDELINES.md).results/ 下的配对结果文件为输入。Presidio 走全量语料（CPU）；OBI（CPU）与开源 LLM（Qwen-9B、Gemma-31B，GPU）取每 benchmark 120 篇子样本。等价分析是纯再分析（不做模型推理）。§7 的替身质量类型学由 200 条的 AI review 定量（非人工验证；标签在 data/annotation/surrogate_quality_audit.csv）；离线工具 annotate.html 支持可选的人工双标注（指南见 data/annotation/GUIDELINES.md）。
All benchmarks are public or synthetic (ASQ-PHI synthetic; MEDDOCAN shared-task; PII-Masking-300k synthetic; MultiCoNER v2 public). No real patient data. Respect each source dataset's original license.所有 benchmark 均为公开或合成数据（ASQ-PHI 合成；MEDDOCAN 评测任务；PII-Masking-300k 合成；MultiCoNER v2 公开）。不含真实病患数据。请遵守各源数据集的原始许可。