Practical Guide: Using Claude for Preliminary Source Analysis While Avoiding Hallucinations

Last week, we explored how modern AI systems are essentially sophisticated pattern-matching machines—like Clever Hans, the mathematical horse who appeared to understand numbers but was actually responding to subtle cues. Today, we'll deepen that metaphor and explore practical techniques for working with these well-intentioned but sometimes overconfident assistants.

The Eager-to-Please Assistant

What made Clever Hans so compelling wasn't just his apparent mathematical abilities, but his eagerness to provide answers. When asked a question, Hans would tap his hoof with conviction—even when he didn't truly understand what was being asked. His trainer hadn't taught him to say "I don't know"; he'd been rewarded for producing answers, not for acknowledging uncertainty.

Modern AI systems like Claude operate under similar constraints. They've been extensively trained to produce helpful, coherent responses rather than to express confusion or uncertainty. When presented with a question at the edge of their knowledge or a document with ambiguous information, these systems don't naturally say, "I can't determine that from the available information." Instead, like Clever Hans eagerly tapping his hoof, they generate what seems like a reasonable answer based on the patterns they've observed.

This eager-to-please quality—the tendency to confidently provide outputs even when knowledge is incomplete—creates the hallucination problem that investigators must carefully navigate.

The Hallucination Problem

AI hallucinations aren't random errors or "glitches" in the system—they're direct consequences of how these models are designed to function. When Claude encounters a question it doesn't have complete information to answer, it doesn't have a built-in mechanism to recognize knowledge boundaries the way humans do. Instead, like Clever Hans sensing his questioner's subtle body language, Claude detects patterns in the prompts and data it's given and produces what it predicts is the most appropriate response.

Consider what happens when we ask about a person mentioned in a document:

"What role did Michael Chen play in the company restructuring?"

If the document contains only passing mentions of Chen without specifying his role, Claude won't simply say "The document doesn't specify Chen's role." Instead, drawing on patterns from its training data about company restructurings and typical roles, it might confidently state that "Michael Chen served as the transition coordinator during the company restructuring, overseeing the consolidation of redundant departments." This sounds plausible and specific—exactly what a helpful assistant should provide—but it's entirely fabricated.

For investigators, this creates scenarios where fabricated details appear alongside accurate information without clear differentiation:

• Nonexistent dates or figures that "fill gaps" in timelines

• Fabricated relationships between entities based on typical patterns

• Invented details about transactions or events to create coherent narratives

• False attributions or quotations that seem authentic but never appeared in the source

Practical Techniques for Hallucination-Resistant Source Analysis

1. Structure Your Inputs as Explicit Tasks

INEFFECTIVE: "What can you tell me about this document?"

EFFECTIVE: "Identify all companies mentioned in this document. For each company, list only the following: (1) full legal name as written, (2) registration location if specified, and (3) any direct connections to individual names mentioned."

By constraining the task, you limit the opportunity for hallucination and create clear accountability for verification.

2. Use Explicit Citation Requirements

When asking Claude to analyze text, require it to indicate where information comes from:

"For each claim or observation you make, specify the exact page number, paragraph, or section of the document where this information appears. If you cannot find a specific source for a claim within the document, explicitly mark it as 'Not Found in Document.'"

This creates a built-in verification system where claims without citations become immediately suspect.

3. Separate Extraction from Analysis

The hallucination risk increases dramatically when AI systems are asked to simultaneously extract and interpret information. Instead, use a two-stage approach:

Stage 1: "Extract all dates and associated events mentioned in this document, using exact quotes where possible."

Stage 2: (After human verification) "Based on the verified timeline I've confirmed, identify any patterns or anomalies in the sequence of events."

4. Implement the "Extract-then-Ask" Method

Rather than asking open-ended questions about documents, first have the AI extract specific elements, which you can verify before proceeding:

1. "Extract all references to offshore transactions in this document."

2. Verify these extractions against the source material

3. "Using only the verified extractions I've confirmed, summarize the pattern of offshore movements."

5. Use Triangulation Across Multiple Prompts

A powerful technique for detecting potential hallucinations is to approach the same question from multiple angles:

• "List all individuals mentioned as directors in this document."

• "For each company in the document, list any associated directors."

• "Extract all sentences that mention directors."

Inconsistencies across these responses often reveal where the AI system is filling in gaps rather than reporting what's actually in the text.

A Real-World Example: The Redacted Document Challenge

To illustrate these techniques in action, let's consider a recent case where our team needed to analyze a heavily redacted regulatory filing:

Original approach: An investigator asked Claude to "summarize the key allegations in this document." The result was a coherent narrative that seemed plausible—until we discovered it had invented several details to fill gaps created by redactions.

Improved approach: We restructured our prompt:

Task: Analyze the redacted document with the following constraints: 
1. Identify all visible allegations with corresponding page numbers. 
2. Mark any section where redactions create uncertainty with [REDACTION GAP].
3. Do not attempt to infer what might be in redacted sections. 
4. If a sentence is interrupted by redaction, indicate this explicitly.

The result was messier but far more reliable—it clearly distinguished between what was actually visible in the document and what was obscured, preventing us from building analysis on hallucinated details.

Key Principles to Remember

1. Verification remains essential - AI output should always be cross-checked against the source material

2. Specificity reduces hallucination - Narrow, concrete tasks generate more reliable responses

3. Explicitness about uncertainty - Train the AI to explicitly mark uncertainty rather than make confident guesses

4. Process over product - Focus on creating reliable analytical processes rather than seeking polished outputs

5. Respect the eager-to-please nature - Remember that systems like Claude are designed to provide answers, not to admit knowledge gaps

The Clever Hans metaphor reminds us that modern AI systems aren't deliberately deceptive—they're simply doing what they've been trained to do: provide helpful responses based on pattern recognition. The horse wasn't intentionally misleading his audience; he was earnestly trying to please. Similarly, Claude isn't trying to trick investigators with hallucinations; it's trying to be as helpful as possible by providing coherent, complete-seeming answers.

By designing our investigative approaches with this understanding, we can harness the remarkable pattern-matching capabilities of these systems while implementing the guardrails necessary to account for their fundamental limitations.