Material that can be identifying even when it does not look like PII
What risk feels like
Not "will the model be clever?"
More like: "will this create a professional conduct problem?"
Or: "will this accidentally disclose client confidential information?"
Legal users do not think like IT admins. They think like risk managers with billable hours.
Where our use case fits
Enterprise AI plans are a contractual trust solution. Reversible de-identification is a structural trust solution.
Contractual trust
The vendor promises security, isolation, no training, retention limits, and compliance controls.
Lower friction. Better context. Still requires trust in the vendor, tenant, admin controls, subprocessors, and data handling terms.
Structural trust
The system prevents raw sensitive text from crossing a boundary in the first place.
More friction. Harder engineering. But if the LLM only sees [PERSON_1], there is less raw identity to leak.
Working definition: LLM means the downstream model/API. Application architecture means the controls around it: ingestion, review, mappings, storage, logs, and boundaries.
This is not either-or. Good legal AI will often need both: strong contracts and safer architecture.
Not the whole privacy stack
This talk sits at the application boundary, not across the whole privacy-tech stack.
Broader privacy-preserving AI techniques include:
Private models and private inference
Trusted execution environments
Differential privacy
Multi-party computation and federated learning
Homomorphic encryption
Thanks to Nick Lothian for the broader privacy-tech framing. My slice is narrower: detect, transform, review, send only de-identified payloads, then re-identify safely.
Practically, a five-person family law firm is not likely to implement homomorphic encryption in a week. They need a concrete boundary they can understand and adopt first.
The naive version
Seems simple to build.
Detect PII
>
Replace it
>
Send to LLM
>
String replace back
Why it feels done
Clean demos look great
Regexes and NER find obvious things
The diff looks reassuring
Actual lawyers say
"That nickname identifies the client."
"This organisation identifies the matter."
"Will it remember my correction next time?"
"Can I prove what was sent to the LLM?"
So we tested with lawyers using synthetic data and lawyer-nominated meeting transcripts for evals: missed entities, false positives, mapping stability, and safe re-identification.
Failure mode 1
Over-redaction protects privacy, then quietly destroys usefulness.
Original
"John from Acme met Sarah at the Collins Street office before the mediation."
The model can reason about roles, sequence, and context.
Naively scrubbed
"PERSON from ORGANISATION met PERSON at ADDRESS before EVENT."
Now the structure has collapsed. The LLM cannot tell who matters, or why.
Privacy is not just removing information. It is preserving the minimum useful structure for the downstream task to still work.
Failure mode 2
Technically, we removed the name. Practically, we failed to remove the person.
The same person appeared three ways
Gary Palmer > PERSON_1
G. Palmer > another surface form
Gazza > nickname only a human caught
The name was not the only identifier
Forklift + roller door + warehouse move
"There are fifty people in this company and everyone knows who that is."
The lawyer feedback became matter-level context: these aliases refer to the same person.
That was the pivot: privacy was not just entity detection. It was trust, correction, memory, and context.
The pivot
The architecture improved when we stopped asking "did we mask it?" and asked four harder questions.
What was detected?
Entity, span, confidence, source rule.
What replaced it?
Stable placeholder, mapping, scope.
Who decided?
Model, pattern, custom rule, or human override.
Can we safely reverse it later?
Across edits, reruns, exports, and downstream AI outputs.
Key design choice
Placeholders are not just strings. They are contracts.
Why neutral tokens
They avoid leaking relationships like CLIENT_FATHER
They avoid leaking scale or industry like MAJOR_BANK_1
They are simple enough for lawyers to inspect
They make the mapping explicit and auditable
What makes them work
Stable within the matter
Typed by entity class
Backed by a persistent mapping
Reviewed and corrected by the user
"[PERSON_1] emailed [PERSON_2] before the meeting at [LOCATION_1]. Later, [PERSON_2] called [ORG_1]'s solicitor."
The trade-off: neutral tokens preserve less semantic context, so the surrounding system has to carry stability, review, and re-identification.
Why reversibility changes everything
You are no longer storing raw text and scrubbed text.
First-class objects
Entities and spans
Mappings and placeholder IDs
Document revisions and scrub versions
Manual overrides and source-of-truth flags
Provenance and audit history
Normal legal cases
User edits a placeholder sentence
User pastes fresh raw text
Summary contains only some placeholders
One person appears under several names
Two people share one name
The mapping is not metadata. It is the product's memory.
Human control
Manual review is not a fallback. It is part of the architecture.
Users need to...
Add a missed entity
Mark something as safe
Add custom family-law terms
Decide what is sensitive in their matter
The system needs to know...
Does manual safe beat future detection?
Does a custom pattern beat the model?
How do overlaps resolve?
Will it remember the lawyer's judgement?
If the system forgets the user's judgement, it feels like it is arguing with them.
The architecture
The LLM should not be your privacy boundary. The application architecture should be.
Raw text controlled zone
>
Detection patterns + model + rules
>
Review UI built with lawyers
>
De-identified payload only this crosses
Application privacy boundary
Secure API + LLM per tenant, used by Smartnote
>
Re-identification layer
>
Final output
Mapping store isolated, versioned, auditable
While prototyping with lawyers, we built both the review UI and a secure per-tenant API that Smartnote calls from its own workflow.
Recorded demo
A short walkthrough, without the demo gods.
De-identifyReviewSend scrubbed text downstreamRe-identify final output
What changed
What we stopped doing
Assuming more redaction is always better
Hiding uncertainty from the user
Treating re-identification as string replace
Trusting one-shot transformations
Thinking this was just an NLP feature
Putting the model on the privacy boundary
De-identification in production is a systems engineering problem: UI, data model, persistence, model boundary, logging, tests, and trust.
Takeaways
If you build AI around sensitive text, here is the checklist.
Design for reversibility from day one
Make mappings first-class objects
Preserve useful structure, not raw identity
Assume users will edit everything
Treat manual overrides as core data
Do not rely on the model to enforce privacy
Optimize for trust recovery, not just accuracy
Make it obvious what crossed the boundary
Close
Most AI de-identification fails in production because teams treat it as a masking step.
In real legal workflows, it is an architecture for controlled transformation: preserve meaning without preserving identity, allow automation without removing judgement, and stay reversible without becoming fragile.
The LLM should not be your privacy boundary. The application architecture should be.
Find me after
Moin Zaman
Working on
SmartScrub and Smartnote, AI systems for sensitive legal workflows.