How to Detect and Remove Contradictory Memories
Before You Start
Contradiction detection is most effective as part of a consolidation pipeline, because the clustering step in consolidation already identifies groups of related memories that are likely to contain contradictions. If you have not built a consolidation pipeline, you can still run contradiction detection as a standalone process, but you will need to build the candidate identification step separately.
You need either an LLM or a specialized NLI (natural language inference) model to compare factual claims. LLMs are more flexible and can handle nuanced contradictions, but they are more expensive per comparison. NLI models like DeBERTa fine-tuned on MNLI can classify pairs of statements as entailment, contradiction, or neutral at a fraction of the cost. Choose based on your accuracy requirements and budget.
Step-by-Step Implementation
Not every pair of memories needs to be checked for contradictions. Focus on pairs that share entities and cover the same topic, because contradictions can only occur between memories that make claims about the same subject. Use entity overlap (two or more shared entities) and high semantic similarity (above 0.70 cosine similarity) to identify pairs worth comparing. This filtering step reduces the comparison space from O(n^2) to a manageable set of targeted comparisons.
def find_contradiction_candidates(memories, entity_overlap_min=2,
sim_threshold=0.70):
candidates = []
for i, ma in enumerate(memories):
for mb in memories[i+1:]:
shared = set(ma['entities']).intersection(mb['entities'])
if len(shared) < entity_overlap_min:
continue
sim = cosine_similarity(ma['embedding'], mb['embedding'])
if sim >= sim_threshold:
candidates.append((ma, mb, shared))
return candidatesParse each memory into discrete factual assertions. A memory that says "the API uses OAuth 2.0 for authentication and rate limits to 100 requests per minute" contains two claims: one about the authentication method and one about the rate limit. Extracting claims individually lets you detect partial contradictions where one claim conflicts while others are consistent. Use an LLM prompt to extract claims as structured data.
def extract_claims(memory_content):
prompt = f"""Extract discrete factual claims from this text.
Return each claim as a JSON object with 'subject', 'predicate',
and 'object' fields.
Text: {memory_content}
Return a JSON array of claims."""
response = llm_call(prompt)
return parse_json(response)For each pair of candidate memories, compare their extracted claims to find conflicts. Two claims contradict when they share the same subject and predicate but assert different objects. "The API rate limit is 100 requests per minute" contradicts "The API rate limit is 500 requests per minute" because they make different assertions about the same property of the same subject. Complementary claims about the same subject, such as "the API uses OAuth" and "the API supports rate limiting," are not contradictions even though they share the subject.
def detect_contradictions(claims_a, claims_b):
contradictions = []
for ca in claims_a:
for cb in claims_b:
if ca['subject'].lower() == cb['subject'].lower():
if ca['predicate'].lower() == cb['predicate'].lower():
if ca['object'].lower() != cb['object'].lower():
contradictions.append({
'claim_a': ca,
'claim_b': cb,
'subject': ca['subject'],
'predicate': ca['predicate']
})
return contradictionsWhen a contradiction is detected, determine which claim to preserve. The default priority order is: recency first (the more recently stored or updated memory is more likely to reflect current reality), then confidence (higher confidence indicates better corroboration), then corroboration count (more independent confirmations indicate higher reliability). For domains where recency is not the strongest signal, such as legal or medical knowledge, configure confidence as the primary resolution criterion instead.
def resolve(mem_a, mem_b, strategy='recency_first'):
if strategy == 'recency_first':
time_a = mem_a.get('updated_at', mem_a['created_at'])
time_b = mem_b.get('updated_at', mem_b['created_at'])
if time_a != time_b:
return (mem_a, mem_b) if time_a > time_b else (mem_b, mem_a)
if strategy == 'confidence_first' or True:
conf_a = mem_a.get('confidence', 5.0)
conf_b = mem_b.get('confidence', 5.0)
if conf_a != conf_b:
return (mem_a, mem_b) if conf_a > conf_b else (mem_b, mem_a)
corr_a = mem_a.get('corroboration_count', 1)
corr_b = mem_b.get('corroboration_count', 1)
return (mem_a, mem_b) if corr_a >= corr_b else (mem_b, mem_a)After resolution, handle the losing memory. You have three options depending on severity. For clear contradictions where the losing claim is definitely wrong, remove the contradicted content from the memory entirely. If the memory contains other valid claims alongside the contradicted one, edit the memory to remove only the conflicting claim and preserve the rest. For ambiguous contradictions where the resolution is less certain, demote the losing memory's confidence score rather than deleting content, so it still exists in the store but ranks lower in retrieval results.
Every contradiction resolution should be logged with the IDs of both memories, the specific claims that conflicted, the resolution strategy used, and the outcome. This audit trail is essential for debugging incorrect resolutions, tuning resolution strategies over time, and satisfying compliance requirements that may mandate explainability for automated data decisions. Store the log outside the memory store itself, in a dedicated audit table or log file.
def log_resolution(winner, loser, contradiction, strategy, action):
entry = {
'timestamp': time.time(),
'winner_id': winner['id'],
'loser_id': loser['id'],
'contradiction': contradiction,
'strategy': strategy,
'action': action, # 'removed', 'demoted', or 'edited'
'winner_confidence': winner.get('confidence'),
'loser_confidence': loser.get('confidence')
}
audit_log.append(entry)Common Contradiction Patterns
The most frequent contradiction pattern is temporal supersession, where a newer fact replaces an older one. "We use Python 3.8" contradicted by "We upgraded to Python 3.12" is a typical example. Recency-first resolution handles these correctly in almost all cases.
The second pattern is source disagreement, where different sources provide conflicting information about the same subject. "The documentation says the limit is 100" contradicted by "the team lead said the limit is 500." Confidence and corroboration-based resolution works better for this pattern, because the more confirmed claim is more likely correct regardless of which was stored more recently.
The third pattern is scope ambiguity, where two memories appear to contradict but actually apply to different contexts. "The staging API uses HTTP" and "the production API uses HTTPS" look like a contradiction about the API protocol, but they describe different environments. Entity extraction and claim comparison help distinguish these from true contradictions, but some scope ambiguity will require human review.
Contradiction detection runs automatically during consolidation. Every memory in your store stays consistent and current.
Get Started Free