How to Detect and Redact PII in LLM Inputs and Outputs
PII in LLM applications flows in two directions, and each direction creates a different risk. When a user pastes a customer record containing a social security number into a chat with a model hosted by OpenAI or Anthropic, that SSN has been transmitted to a third party and potentially logged, cached, or stored according to the provider's data retention policies. The privacy violation happens at transmission time, before the model even processes the data. Input-side PII redaction prevents this by detecting and replacing sensitive data before the API call is made.
When a model generates a response that includes a real phone number, email address, or home address, that data may have been memorized from the model's training corpus. Large language models trained on web data have been demonstrated to reproduce personal information that appeared in their training set, including names, addresses, phone numbers, and email addresses of real individuals. Output-side PII redaction catches this by scanning the model's response before it reaches the user, regardless of where the PII came from.
Step 1: Define Your PII Categories
PII is not a single category. It encompasses dozens of data types, and the specific types that matter for your application depend on your domain, user base, geography, and regulatory requirements. Start by categorizing PII into tiers based on sensitivity and detection approach.
High-sensitivity structured PII has predictable formats and is detectable with regex patterns. This includes social security numbers (US format: NNN-NN-NNNN), credit card numbers (13-19 digits with specific prefix patterns for Visa, Mastercard, Amex), bank account and routing numbers, passport numbers, driver's license numbers (format varies by state/country), and medical record numbers. These are the most damaging if leaked and the easiest to detect programmatically.
Medium-sensitivity structured PII also has somewhat predictable formats but produces more false positives. This includes phone numbers (many national formats, often confused with other number sequences), email addresses (well-defined format but appear in many legitimate contexts), IP addresses (IPv4 and IPv6 formats), dates of birth (many date formats, often confused with other dates), and physical addresses (semi-structured, highly variable format). Detection requires more nuanced logic to distinguish genuine PII from coincidental pattern matches.
Unstructured PII does not follow fixed patterns and requires natural language understanding to detect. This includes personal names, employer names, school names, location references that identify a person ("my apartment at 5th and Main"), medical conditions mentioned in context ("my diabetes diagnosis"), financial situations ("I owe $50,000 on my mortgage"), and relationship details ("my ex-husband John"). Unstructured PII is the hardest to detect and the most common in conversational AI applications because users naturally include personal context in their messages.
Your regulatory environment determines which categories are mandatory. GDPR covers all data that can identify a natural person, which is very broad. HIPAA covers protected health information, which includes medical record numbers, diagnosis codes, treatment details, and any data that could link to a patient's health records. CCPA covers personal information of California residents. PCI DSS covers payment card data specifically. Map your regulatory requirements to specific PII categories and prioritize those categories in your detection pipeline.
Step 2: Build Regex Detection for Structured PII
Regex-based detection is the first layer because it is fast (microseconds per check), deterministic (no model inference required), and highly precise for structured data types. Build a library of patterns for each structured PII category you identified.
Social Security Numbers: The US SSN format is three digits, a hyphen, two digits, a hyphen, and four digits. Your regex should match both hyphenated (123-45-6789) and non-hyphenated (123456789) formats. Apply validation rules: the first three digits (area number) cannot be 000, 666, or 900-999. The middle two digits (group number) cannot be 00. The last four digits (serial number) cannot be 0000. These validation rules reduce false positives from random nine-digit sequences that happen to match the format.
Credit Card Numbers: Match the major card network patterns. Visa starts with 4 and has 13 or 16 digits. Mastercard starts with 51-55 or 2221-2720 and has 16 digits. American Express starts with 34 or 37 and has 15 digits. Discover starts with 6011, 644-649, or 65 and has 16 digits. Apply the Luhn algorithm (checksum validation) to reduce false positives from random number sequences. Account for spaces, hyphens, and no separators between digit groups.
Phone Numbers: Phone number detection requires handling multiple national formats. For US numbers, match (NNN) NNN-NNNN, NNN-NNN-NNNN, NNN.NNN.NNNN, and NNNNNNNNNN formats, with optional +1 country code. For international numbers, match the E.164 format (+[country code][number]) and common national formats for your target geographies. Phone numbers produce the most false positives of any structured PII type because many legitimate number sequences (order numbers, reference codes, product IDs) contain 7-11 digit sequences that resemble phone numbers. Context-aware detection (checking surrounding text for words like "call," "phone," "text," "mobile") reduces false positives significantly.
Email Addresses: The RFC 5322 email format is well-defined and detectable with a single regex. Match [local-part]@[domain].[tld] with appropriate character class restrictions. Email addresses produce relatively few false positives because the @-sign and domain structure are distinctive. The main consideration is whether to redact emails that are clearly business/organizational (support@company.com) or only personal emails, depending on your privacy requirements.
IP Addresses: IPv4 addresses match the NNN.NNN.NNN.NNN pattern with each octet in the 0-255 range. IPv6 addresses use colon-separated hexadecimal groups. IP addresses in LLM context are relatively rare but can identify individuals when combined with timestamps or other data.
Step 3: Add NER-Based Detection for Unstructured PII
Named entity recognition (NER) models identify entities like person names, organizations, locations, dates, and other categories in free text. A NER model catches PII that no regex can detect: "My name is Sarah Chen and I work at Meridian Health" contains two PII entities (person name and employer) that are identifiable only through language understanding.
Microsoft's Presidio is the most widely used open-source PII detection framework. It combines regex-based recognizers for structured PII with NER-based recognizers for unstructured PII, and it provides a unified API that returns all detected PII entities with their types, positions, and confidence scores. Presidio supports multiple NER backends including spaCy, Stanza, and transformer-based models, letting you trade off between speed and accuracy.
SpaCy's NER models provide fast, accurate entity detection for the most common categories: PERSON, ORG, GPE (geopolitical entity), DATE, and MONEY. The en_core_web_lg model runs at approximately 10,000 words per second on a CPU and provides good accuracy for English text. For multilingual applications, spaCy offers models for dozens of languages. The limitation is that spaCy's default categories do not cover all PII types; you may need to fine-tune a custom model or add post-processing rules to detect domain-specific entities like medical conditions or financial account references.
Transformer-based NER models (fine-tuned BERT, RoBERTa, or similar architectures) provide higher accuracy than spaCy, especially on complex or ambiguous text, at the cost of slower inference (100-500 words per second on CPU, significantly faster on GPU). For applications where PII detection accuracy is critical and latency budget allows it, a transformer model is the better choice. HuggingFace hosts several pre-trained PII detection models that can be deployed directly or fine-tuned on your domain data.
Contextual analysis improves detection of ambiguous entities. The string "Jordan" could be a person name, a country, or a brand name depending on context. A context-aware detector considers surrounding words: "my friend Jordan" suggests a person, "shipped from Jordan" suggests a country. Building context rules for the most common ambiguous entities in your domain reduces both false positives and false negatives.
Step 4: Implement Redaction with Typed Placeholders
When PII is detected, replace it with a typed placeholder token that preserves the semantic role of the redacted data without exposing the actual value. Typed placeholders are better than a generic [REDACTED] marker because they maintain the coherence of the text for the model's processing and make the redaction transparent to the user.
Use consistent placeholder formats: [PERSON_NAME_1], [SSN_1], [EMAIL_1], [PHONE_1], [ADDRESS_1]. The type label identifies what was redacted, and the numeric suffix distinguishes multiple entities of the same type in the same message. If a user mentions two different people, the redacted text reads "I spoke with [PERSON_NAME_1] and [PERSON_NAME_2] about the issue" rather than "I spoke with [REDACTED] and [REDACTED] about the issue."
For input-side redaction, decide whether redaction should be reversible or irreversible. Reversible redaction stores a mapping between placeholders and original values in your session store, allowing the application to reconstruct the original message for logging, audit trails, or response personalization. Irreversible redaction discards the original values after replacement, providing stronger privacy guarantees but losing the ability to reference the original data. Your regulatory requirements and use case determine the right approach: HIPAA-covered applications often require irreversible redaction for PHI, while customer support applications may need reversible redaction to include customer names in responses.
For output-side redaction, irreversible redaction is almost always the right choice. If the model generates PII from memorized training data, you have no legitimate reason to preserve that data, and redacting it irreversibly is the safe default. The placeholder in the output signals to the user that the model attempted to include sensitive data and the system caught it.
Handle edge cases in redaction carefully. Partial matches (a phone number that is actually a product serial number) should be reviewed against context before redaction. Nested PII (an email address containing a person's name) requires coordinated redaction. PII that is essential to the conversation's purpose (a user asking the model to format their own phone number) may need a different treatment, such as processing the request but not transmitting the actual number to the model API.
Step 5: Apply to Both Input and Output Pipelines
The same detection pipeline serves both input and output scanning, but the response to detection differs.
On the input side, detected PII is redacted before the message is sent to the model API. The model receives the message with placeholders and processes it without ever seeing the sensitive data. This is the primary defense against data exposure to third-party APIs. If you use reversible redaction, the response can be post-processed to replace placeholders with original values before delivery to the user, so the user sees their original names and references in the response even though the model never processed them.
On the output side, detected PII triggers a different workflow. If the model generates what appears to be a real SSN, phone number, or address, the system must decide: redact and deliver the response with placeholders, regenerate the response with an explicit instruction not to include PII, or block the response entirely and deliver a fallback. The right choice depends on severity. A generated SSN or credit card number should trigger a block and regeneration. A generated email address might warrant simple redaction. A generated name that might be from training data might warrant a softer intervention like replacing with a generic reference.
Test your pipeline with both benign and adversarial inputs. Benign tests verify that the pipeline correctly handles legitimate PII in normal conversation contexts. Adversarial tests verify that the pipeline catches PII even when it is obfuscated, encoded, split across multiple messages, or embedded in unusual formats. Build a test suite with at least 100 examples covering each PII category in various formats and contexts, including false positive cases (phone number-like product codes, date-like serial numbers) that should not be redacted.
PII detection for LLM applications requires both regex patterns (for structured data like SSNs and credit cards) and NER models (for unstructured data like names and addresses), applied to both input and output pipelines. Input redaction prevents data exposure to model APIs. Output redaction catches memorized or reconstructed PII in model responses.