PROCUX Verify™: How Evidence-Based AI Slashes Hallucination Risk

Introduction: The AI Trust Problem

Every AI system faces a critical challenge: hallucination—when AI confidently states false information as fact.

Public benchmarks consistently show that standalone AI systems hallucinate at meaningful rates—often double-digit percentages on factual tasks. For businesses, this creates serious risks:

• Legal liability from incorrect advice
• Financial losses from wrong data
• Customer trust erosion
• Compliance violations

PROCUX Verify™ solves this with evidence-based AI: every claim gets verified, every answer gets sourced.

The Hallucination Crisis: Why Traditional AI Fails

The Problem with "Black Box" AI

Traditional AI systems (standalone chatbots and generic assistants) generate responses through neural network inference—predicting what text should come next based on patterns learned from training data.

The fundamental issue: These systems don't "know" if their output is true. They only know what's statistically likely.

Hallucination Examples (Illustrative)

Example 1: Financial Recommendation Gone Wrong

User: "What was our Q3 revenue growth?"
Traditional AI: "Your Q3 revenue grew 23% year-over-year to $4.2M."
Reality: Revenue actually declined 5% to $3.1M.

Result: Executive team made expansion decisions based on false data.
Cost: $200K wasted on premature hiring.

Example 2: Legal Compliance Failure

User: "Do we need GDPR consent for email marketing in Germany?"
Traditional AI: "No, you can email customers without consent under legitimate interest."
Reality: GDPR requires explicit opt-in consent for marketing emails.

Result: €50,000 GDPR fine + customer trust damage.

Example 3: Technical Support Misinformation

User: "How do I reset the API key?"
Traditional AI: "Run the command: reset-api-key --force"
Reality: No such command exists. Correct process requires admin portal access.

Result: Customer attempted non-existent solution, escalated to support (wasted 2 hours).

How PROCUX Verify™ Works: Evidence-Based AI Architecture

PROCUX Verify™ uses a 6-step verification pipeline that transforms every AI response into an evidence-backed, citation-rich answer.

Architecture Overview

@dataclass
class EvidenceBasedResponse:
    """Complete evidence-based response with verification"""
    content: str                          # The actual response text
    confidence: float                     # Overall confidence (0-1)
    citations: List[Citation]             # All citations with sources
    reasoning_chain: List[str]            # Transparent reasoning steps
    verification_status: VerificationStatus  # VERIFIED/PARTIAL/UNVERIFIED
    claims_total: int                     # Total claims extracted
    claims_verified: int                  # Claims with source backing
    claims_unverified: int                # Claims without verification
    processing_time_ms: float             # Verification time

The 6-Step Verification Pipeline

Step 1: Claim Extraction

Every AI response gets decomposed into individual verifiable claims.

6 Claim Types Detected:

1. QUANTITATIVE: Numbers, percentages, metrics ("Revenue grew 23%")
2. CAUSAL: Cause-effect relationships ("Due to market expansion, sales increased")
3. COMPARATIVE: Comparisons ("Better than competitors", "More efficient")
4. TEMPORAL: Time-based statements ("In Q3 2024", "Last month")
5. RECOMMENDATION: Actionable advice ("You should implement", "We recommend")
6. DESCRIPTIVE: Factual statements ("The company has 50 employees")

Technical Implementation:

class ClaimExtractor:
    """Extract verifiable claims from AI responses"""

    async def extract_claims(self, text: str) -> List[Claim]:
        """
        Uses NLP patterns to identify claims

        Process:
        1. Split text into sentences
        2. Skip questions and very short sentences
        3. Classify claim type using regex patterns
        4. Extract entities and keywords
        5. Create Claim object with metadata
        """
        claims = []
        sentences = self._split_sentences(text)

        for i, sentence in enumerate(sentences):
            if len(sentence) < 20 or sentence.endswith('?'):
                continue  # Skip short sentences and questions

            claim_type = self._classify_claim_type(sentence)
            entities = self._extract_entities(sentence)
            keywords = self._extract_keywords(sentence)

            claim = Claim(
                text=sentence,
                claim_type=claim_type,
                position=i,
                entities=entities,
                keywords=keywords
            )
            claims.append(claim)

        return claims

Example Output:

# AI Response: "Your Q3 revenue grew 23% to $4.2M due to market expansion."

# Claims Extracted:
[
    Claim(
        text="Your Q3 revenue grew 23%",
        claim_type=ClaimType.QUANTITATIVE,
        keywords=["revenue", "grew", "23%"],
        entities=["Q3"]
    ),
    Claim(
        text="revenue grew to $4.2M",
        claim_type=ClaimType.QUANTITATIVE,
        keywords=["revenue", "4.2M"],
        entities=[]
    ),
    Claim(
        text="due to market expansion",
        claim_type=ClaimType.CAUSAL,
        keywords=["market", "expansion"],
        entities=[]
    )
]

Step 2: Source Gathering

PROCUX aggregates all available sources for verification.

8 Source Types:

1. COMPANY_DNA: Your company's knowledge base (95% confidence weight)
2. EXECUTIVE_ANALYSIS: Output from other AI executives (80% confidence)
3. WEB_SEARCH: Real-time web data (60% confidence)
4. DOCUMENT: Uploaded PDFs, docs (90% confidence)
5. DATABASE: Direct database queries (98% confidence)
6. MEMORY: Previous conversation context (70% confidence)
7. USER_INPUT: User-provided information (85% confidence)
8. INFERENCE: AI reasoning (50% confidence—requires verification)

Code:

def _gather_sources(
    self,
    task_results: List[Any],        # Other executive outputs
    additional_sources: List[Dict], # Web search, docs
    company_context: Dict           # Company DNA
) -> List[Dict]:
    """Gather all sources for verification"""
    sources = []

    # Add executive analyses
    for executive_result in task_results:
        sources.append({
            "name": f"{executive.upper()} Analysis",
            "type": SourceType.EXECUTIVE_ANALYSIS,
            "content": executive_result.content,
            "confidence": 0.8
        })

    # Add company DNA (highest confidence)
    if company_context:
        sources.append({
            "name": "Company DNA",
            "type": SourceType.COMPANY_DNA,
            "content": str(company_context),
            "confidence": 0.95
        })

    # Add additional sources (web, docs)
    sources.extend(additional_sources)

    return sources

Step 3: Claim Verification

Each claim gets verified against all available sources using a hybrid similarity algorithm.

Verification Algorithm:

async def _calculate_similarity(self, claim: str, source: str) -> float:
    """
    Hybrid verification using 3 methods:

    1. Jaccard Similarity (40% weight): Word overlap
    2. Keyword Matching (40% weight): Important term overlap
    3. Phrase Matching (20% weight): Exact phrase detection
    """
    # Method 1: Jaccard (word-level overlap)
    words_claim = set(re.findall(r'\b\w+\b', claim.lower()))
    words_source = set(re.findall(r'\b\w+\b', source.lower()))
    jaccard = len(words_claim & words_source) / len(words_claim | words_source)

    # Method 2: Keyword matching (weighted by importance)
    keywords_claim = extract_keywords(claim)
    keywords_source = extract_keywords(source)
    keyword_overlap = len(keywords_claim & keywords_source) / max(len(keywords_claim), 1)

    # Method 3: Exact phrase matching
    phrase_match = 0.3 if claim[:30].lower() in source.lower() else 0.0

    # Combine with weights
    final_score = (jaccard * 0.4) + (keyword_overlap * 0.4) + (phrase_match * 0.2)

    return min(final_score, 1.0)

Verification Threshold:

• Similarity > 0.35 = Claim VERIFIED
• Similarity ≤ 0.35 = Claim UNVERIFIED

Example:

# Claim: "Q3 revenue grew 23%"
# Source 1 (Company DNA): "Q3 2024 revenue: $4.2M (23% YoY growth)"
# Similarity: 0.72 ✅ VERIFIED

# Source 2 (Executive Analysis): "Market trends suggest growth"
# Similarity: 0.18 ❌ NOT VERIFIED (too generic)

Step 4: Citation Generation

Verified claims become citations with confidence scores and excerpts.

Citation Structure:

@dataclass
class Citation:
    id: str                    # Unique citation ID (MD5 hash)
    claim: str                 # The claim being cited
    source: str                # Source name
    source_type: SourceType    # Type of source (DNA, web, etc.)
    confidence: float          # 0-1 confidence score
    excerpt: str               # Relevant excerpt from source
    url: Optional[str]         # Source URL if available
    timestamp: datetime        # When citation was created
    metadata: Dict[str, Any]   # Additional context

# Example Citation:
Citation(
    id="a3f2b1c4d5e6",
    claim="Q3 revenue grew 23% to $4.2M",
    source="Company DNA - Financial Records",
    source_type=SourceType.COMPANY_DNA,
    confidence=0.95,
    excerpt="Q3 2024 Financial Summary: Revenue $4.2M (23% YoY growth vs Q3 2023 $3.4M)",
    url="internal://finance/q3-2024-report",
    metadata={"claim_type": "quantitative", "verification_method": "keyword_overlap"}
)

Confidence Boosting:

• Verified claims get confidence boosted by 20% (capped at 100%)
• Unverified claims default to 50% confidence

Step 5: Overall Confidence Calculation

The system calculates an overall confidence score for the entire response.

Formula:

def _calculate_overall_confidence(
    self,
    citations: List[Citation],
    verified_ratio: float  # % of claims verified
) -> float:
    """
    Weighted confidence calculation:

    70% weight: Average citation confidence
    30% weight: Verification ratio
    """
    # Average citation confidence
    avg_citation_conf = sum(c.confidence for c in citations) / len(citations)

    # Combine with verification ratio
    overall = (avg_citation_conf * 0.7) + (verified_ratio * 0.3)

    return round(min(max(overall, 0.0), 1.0), 2)

# Example:
# 8 claims total, 7 verified (87.5% verification ratio)
# Citation confidences: [0.95, 0.92, 0.88, 0.85, 0.90, 0.78, 0.82, 0.50]
# Average citation confidence: 0.825
# Overall confidence: (0.825 * 0.7) + (0.875 * 0.3) = 0.84 (84%)

Verification Status:

• VERIFIED: ≥80% claims verified
• PARTIALLY_VERIFIED: 50-79% claims verified
• UNVERIFIED: <50% claims verified

Step 6: Reasoning Chain Generation

Every response includes a transparent reasoning chain showing how the conclusion was reached.

Example Reasoning Chain:

reasoning_chain = [
    "1. Analyzed user request and identified key requirements",
    "2. Consulted with 3 executives: CFO, CEO, CMO",
    "3. Gathered evidence from 5 verified sources",
    "4. Verified 7 of 8 claims (87% verification rate)",
    "5. Synthesized findings into comprehensive response",
    "6. Overall confidence assessment: 84%"
]

This appears at the bottom of every AI response, giving users full transparency into the decision-making process.

What Verification Looks Like in Practice

Illustrative Scenario: E-Commerce Customer Support

Scenario: A mid-size e-commerce company (500K orders/year) whose AI customer support keeps giving incorrect refund policies.

PROCUX Verify™ Implementation:

• Connected to company policy documents (source: DOCUMENT)
• Integrated with order database (source: DATABASE)
• Added web search for shipping carrier policies (source: WEB_SEARCH)

What changes after rollout:

• Refund-policy answers now carry citations that point to the company's actual policy documents
• Claims that can't be verified against a source get flagged instead of stated as fact
• Escalations drop because the AI resolves questions correctly the first time
• Every response reports its verification status (VERIFIED / PARTIALLY_VERIFIED / UNVERIFIED)

Before verification, an AI assistant would confidently tell customers they could get refunds on final-sale items. With verification, every answer comes with a citation pointing to the actual policy—and unverifiable answers never go out as fact.

PROCUX Verify™ vs Traditional AI: Technical Comparison

Key Insight: PROCUX Verify™ trades a small increase in latency and cost for a dramatic reduction in errors. For business-critical applications, this is a no-brainer.

How to Use PROCUX Verify™: API Examples

Basic Usage

from procux import ProcuxClient

client = ProcuxClient(api_key="your_api_key")

# Make a query with verification enabled
response = await client.query(
    query="What was our Q3 revenue growth?",
    verify=True,  # Enable PROCUX Verify™
    sources=[
        {"type": "company_dna", "context": "financial_data"},
        {"type": "database", "connection": "postgres://finance_db"}
    ]
)

# Response includes evidence
print(response.content)
# "Your Q3 2024 revenue was $4.2M, representing 23% YoY growth compared to Q3 2023 ($3.4M)."

print(response.confidence)
# 0.95 (95% confidence)

print(response.verification_status)
# VerificationStatus.VERIFIED (>80% claims verified)

# View citations
for citation in response.citations:
    print(f"Claim: {citation.claim}")
    print(f"Source: {citation.source} ({citation.confidence:.0%} confidence)")
    print(f"Excerpt: {citation.excerpt}")
    print("---")

# Output:
# Claim: Your Q3 2024 revenue was $4.2M
# Source: Company DNA - Financial Records (95% confidence)
# Excerpt: Q3 2024 Financial Summary: Revenue $4.2M
# ---
# Claim: representing 23% YoY growth
# Source: CFO Analysis (88% confidence)
# Excerpt: Year-over-year growth analysis shows 23% increase from Q3 2023
# ---

# View reasoning chain
for step in response.reasoning_chain:
    print(step)

# Output:
# 1. Analyzed user request and identified key requirements
# 2. Consulted with 2 executives: CFO, CEO
# 3. Gathered evidence from 3 verified sources
# 4. Verified 2 of 2 claims (100% verification rate)
# 5. Synthesized findings into comprehensive response
# 6. Overall confidence assessment: 95%

Advanced: Custom Verification Thresholds

# Set custom verification requirements
response = await client.query(
    query="Should we expand to European markets?",
    verify=True,
    verification_config={
        "min_confidence": 0.8,           # Require 80%+ confidence
        "min_verified_ratio": 0.9,       # Require 90%+ claims verified
        "require_citations": True,       # Must have citations
        "allowed_source_types": [        # Only trust these sources
            "company_dna",
            "database",
            "executive_analysis"
        ]
    }
)

# If requirements not met, response indicates insufficient evidence
if response.verification_status == VerificationStatus.UNVERIFIED:
    print("Insufficient evidence. Requesting human review.")
    print(f"Only {response.claims_verified}/{response.claims_total} claims verified")

Real-Time Claim Verification

# Verify a single claim against your sources
result = await client.verify_claim(
    claim="Our customer churn rate is below 5%",
    sources=[
        {"type": "database", "query": "SELECT churn_rate FROM metrics WHERE month = 'Dec 2024'"}
    ]
)

print(result.verified)      # True/False
print(result.confidence)    # 0-1 confidence score
print(result.source)        # Which source verified it
print(result.excerpt)       # Relevant excerpt

When to Use PROCUX Verify™

✅ Perfect For:

1. Financial Services: Investment advice, portfolio recommendations, compliance queries
2. Legal: Contract review, regulatory compliance, legal research
3. Healthcare: Medical information, treatment protocols, drug interactions
4. Customer Support: Policy questions, refund eligibility, product specifications
5. Executive Decision Making: Strategic planning, market analysis, competitive intelligence
6. Compliance & Audit: SOX, GDPR, HIPAA compliance verification
7. Education & Training: Factual content generation, knowledge testing
8. Technical Documentation: API docs, troubleshooting guides, technical specs

❌ Not Ideal For:

1. Creative Writing: Poetry, fiction, marketing copy (creativity more important than facts)
2. Casual Conversation: Chitchat, personal opinions (no verification needed)
3. Exploratory Brainstorming: Idea generation (constraints reduce creativity)
4. Real-Time Chat (if latency <500ms required): Verification adds 350-850ms

Frequently Asked Questions

Response: "Q3 revenue estimates vary by source. Company DNA reports $4.2M (95% confidence),
while preliminary CFO analysis suggests $4.0M (72% confidence). Awaiting final audit."

Citations:
[1] Company DNA - Financial Records (95% confidence)
[2] CFO Analysis - Preliminary Estimate (72% confidence)

✓ Extracted claim 1/5: "Q3 revenue grew 23%"
✓ Verified against Company DNA (similarity: 0.72)
✓ Citation generated (confidence: 95%)
...

Technical Architecture: Under the Hood

For developers who want to understand the complete implementation:

Data Flow Diagram

User Query
    ↓
[PROCUX Board™] ← 16 AI Executives analyze
    ↓
[Synthesis] ← Best answer selected
    ↓
[PROCUX Verify™]
    ↓
┌─────────────────────────────────────────┐
│ Step 1: Claim Extraction                │
│ - Split into sentences                  │
│ - Classify claim types (6 types)        │
│ - Extract entities & keywords           │
│ → Output: List[Claim]                   │
└─────────────────────────────────────────┘
    ↓
┌─────────────────────────────────────────┐
│ Step 2: Source Gathering                │
│ - Company DNA (95% confidence)          │
│ - Executive analyses (80% confidence)   │
│ - Database queries (98% confidence)     │
│ - Web search (60% confidence)           │
│ → Output: List[Source]                  │
└─────────────────────────────────────────┘
    ↓
┌─────────────────────────────────────────┐
│ Step 3: Claim Verification (Parallel)   │
│ For each claim:                         │
│   - Calculate similarity vs all sources │
│   - Jaccard (40%) + Keywords (40%) +    │
│     Phrase Match (20%)                  │
│   - Threshold: >0.35 = VERIFIED         │
│ → Output: List[ClaimVerification]       │
└─────────────────────────────────────────┘
    ↓
┌─────────────────────────────────────────┐
│ Step 4: Citation Generation             │
│ - Create Citation objects               │
│ - Extract relevant excerpts             │
│ - Boost verified confidence +20%        │
│ → Output: List[Citation]                │
└─────────────────────────────────────────┘
    ↓
┌─────────────────────────────────────────┐
│ Step 5: Confidence Calculation          │
│ - Average citation confidence (70%)     │
│ - Verification ratio (30%)              │
│ - Determine status (VERIFIED/PARTIAL)   │
│ → Output: float (0-1)                   │
└─────────────────────────────────────────┘
    ↓
┌─────────────────────────────────────────┐
│ Step 6: Reasoning Chain                 │
│ - Build transparent 6-step chain        │
│ - Show which executives consulted       │
│ - Display verification stats            │
│ → Output: List[str]                     │
└─────────────────────────────────────────┘
    ↓
EvidenceBasedResponse (with citations)
    ↓
User sees verified answer + sources

Conclusion: Building Trustworthy AI

The AI trust problem isn't going away. As AI systems make more business-critical decisions, hallucination errors become increasingly costly.

PROCUX Verify™ solves this with:

• ✅ Dramatic error reduction (every claim checked against real sources)
• ✅ Automatic citations (every claim gets sourced)
• ✅ Transparent reasoning (6-step chain)
• ✅ Confidence scores (know when to trust)
• ✅ Full audit trail (compliance-ready)

Bottom line: For business applications where accuracy matters, evidence-based AI isn't optional—it's essential.

Next Steps

1. Try PROCUX Verify™: Start free trial (1,000 verified queries/month)
2. Read the docs: API documentation
3. Explore integration: Connect your data sources
4. See it in action: Live demo

Questions? Contact our team for a personalized implementation plan.

Related Resources

• PROCUX Board™: Multi-Agent Consensus Mechanism
• Progressive Disclosure: How PROCUX Saves 80-90% on AI Costs
• PROCUX DNA™: Teaching AI Your Company Culture
• Evidence-Based AI API Reference

Share this article: Twitter | LinkedIn | Facebook