mantisfury/ArkhamMirror: Local-First AI Platform for Document Intelligence

mantisfury/ArkhamMirror: Local-First AI Platform for Document Intelligence

Investigative work generates overwhelming document volume—PDFs, emails, images, social media^{↗ Bright Coding Blog} archives—that must be parsed, cross-referenced, and analyzed under tight deadlines. Most tools force trade-offs: cloud platforms expose sensitive source material, while local tools lack the AI-assisted extraction and structured analytic techniques that speed up pattern recognition. For journalists, legal advocates, and researchers handling sensitive materials, this creates a serious operational security problem.

mantisfury/ArkhamMirror (also referred to as SHATTERED in its documentation) addresses this directly. It's a local-first, AI-powered document intelligence platform built around a modular "shard" architecture that keeps data on-machine unless explicitly configured otherwise. With 443 GitHub stars, 37 forks, and active development through January 2026, it represents a mature open-source approach to investigative document analysis that prioritizes data sovereignty alongside analytical depth.

What is mantisfury/ArkhamMirror?

mantisfury/ArkhamMirror is a Python^{↗ Bright Coding Blog} 3.10+ platform with a React^{↗ Bright Coding Blog} 18/TypeScript frontend, structured as 27 packages: a core "Frame" providing 17 services, a UI "Shell," and 26 functional shards covering ingestion, analysis, visualization, and export. The project is maintained by Justin McHugh, released under the MIT License, and explicitly designed for investigative journalism, OSINT work, legal self-advocacy, and intelligence analysis workflows.

The platform's architectural philosophy is called "Voltron"—self-contained modules that combine into a unified application without direct dependencies on each other. Shards communicate through an event bus, each operates in its own PostgreSQL^{↗ Bright Coding Blog} schema, and the Frame remains immutable—shards depend on it, never vice versa. This design enables graceful degradation: the system functions with or without GPU acceleration, with or without cloud LLM access, and with or without specific shards installed.

The "local-first" positioning is substantive, not marketing. The platform supports fully air-gapped deployment, runs on PostgreSQL 14+ with pgvector as its only infrastructure dependency, and integrates with local LLM servers (LM Studio, Ollama, vLLM) rather than requiring cloud APIs. When cloud services are used—OpenAI, Groq, or custom endpoints—they're optional one-click configurations, not architectural requirements.

Key Features

AI-Assisted Structured Analysis

The platform implements formal intelligence methodologies: Analysis of Competing Hypotheses (ACH) with matrix scoring, premortem analysis, and "devil's advocate" mode; contradiction detection across documents with severity scoring; pattern recognition for recurring behaviors, temporal sequences, and statistical correlations; and anomaly detection with LLM-powered contextual interpretation. An "AI Junior Analyst" service provides cross-shard analysis for anomaly detection, insight synthesis, and credibility assessment.

Document Processing Pipeline

Ingestion supports PDF, DOCX, images, HTML, and TXT with batch processing and duplicate detection. OCR uses PaddleOCR by default, with optional Vision LLM fallback (local Qwen-VL or cloud GPT-4o/Claude) for complex documents. Parsing offers 8 chunking strategies including semantic and sentence-based approaches. Entity extraction uses spaCy NER (PERSON, ORG, GPE, DATE, etc.) with relationship detection and duplicate merging.

Advanced Visualization

The Graph shard provides 10+ visualization modes: force-directed, hierarchical, circular, Sankey, matrix, geographic (Leaflet-based), causal, argumentation, link analysis (i2 Analyst Notebook-style), and temporal evolution. Analytics include centrality measures (PageRank, betweenness, closeness), community detection, path finding, and cycle detection. The Timeline shard handles temporal event extraction, date normalization across formats, conflict detection, and gap analysis.

Search & Export

Search combines pgvector semantic similarity, PostgreSQL full-text keyword search (BM25), and configurable hybrid weighting. Export supports JSON, CSV, PDF, and DOCX; the Reports shard generates investigation summaries, entity profiles, and ACH reports; Letters produces FOIA requests and legal correspondence from Jinja2 templates; Packets creates versioned investigation bundles with access control.

Media Forensics

A dedicated shard provides EXIF extraction, Error Level Analysis (ELA), perceptual hashing, C2PA verification, and reverse image search integration (TinEye, Google Vision, SerpAPI).

Use Cases

Investigative Journalism & OSINT

Journalists can archive social media content, extract entities and relationships, map information networks, and track source credibility using the MOM/POP/MOSES/EVE deception detection checklists. The ACH matrix structures hypothesis evaluation, while timeline construction and contradiction detection support fact-checking workflows. FOIA request templates and deadline tracking fit public records investigations.

Legal Self-Advocacy

Tenant defense, employment disputes, and consumer protection cases benefit from violation chronology construction, evidence packet assembly, and demand letter generation. The provenance tracking shard maintains evidence chains for court admissibility. Pattern detection identifies recurring landlord behaviors or employer violations across multiple documents.

Healthcare Self-Advocacy

Patients managing chronic conditions or insurance appeals can parse lab results, track symptom progression, and organize treatment timelines. The platform's local-first design is particularly relevant here—medical records stay on-device, addressing HIPAA concerns that cloud tools complicate.

Government Oversight & Civic Engagement

Meeting minutes parsing, vote tracking, campaign finance donor mapping, and policy document comparison support accountability journalism and advocacy. The graph visualization modes expose money flows and stakeholder relationships.

Intelligence Analysis

The structured analytic techniques—ACH, link analysis, temporal pattern detection—map directly to intelligence community workflows. Air-gap deployment support enables classified or sensitive environment operation.

Installation & Setup

Prerequisites

• Python 3.10+
• Node.js 18+ (for local UI development only)
• PostgreSQL 14+ with pgvector extension

Manual Installation

# Clone the repository
git clone https://github.com/yourusername/SHATTERED.git
cd SHATTERED

# Install the Frame
cd packages/arkham-frame
pip install -e .

# Install all shards (or select specific ones)
for dir in ../arkham-shard-*/; do
  pip install -e "$dir"
done

# Install spaCy model
python -m spacy download en_core_web_sm

# Install UI dependencies
cd ../arkham-shard-shell
npm install

The Frame auto-discovers installed shards through Python entry points. The for loop installs all 26 shards; replace with individual paths for selective installation.

Configuration

Create a .env file:

# Required - PostgreSQL with pgvector extension
DATABASE_URL=postgresql://user:pass@localhost:5432/shattered

# Optional - LLM Integration (OpenAI-compatible endpoint)
LLM_ENDPOINT=http://localhost:1234/v1
LLM_API_KEY=your-api-key

# Optional - Embedding Model (default: all-MiniLM-L6-v2)
EMBED_MODEL=all-MiniLM-L6-v2

# Optional - Vision LLM for OCR
VLM_ENDPOINT=http://localhost:1234/v1

# Optional - Auth (required for production)
AUTH_SECRET_KEY=generate-with-openssl-rand-hex-32

Running

# Terminal 1: Start the Frame API (auto-discovers installed shards)
python -m uvicorn arkham_frame.main:app --host 127.0.0.1 --port 8100

# Terminal 2 (optional): Start the UI for development
cd packages/arkham-shard-shell
npm run dev

# Workers start automatically with the Frame
# Background jobs use PostgreSQL SKIP LOCKED pattern

Docker^{↗ Bright Coding Blog} Deployment (Recommended)

# Copy environment template
cp .env.example .env

# Generate a secure auth key
python -c "import secrets; print('AUTH_SECRET_KEY=' + secrets.token_urlsafe(32))"
# Add the output to your .env file

# Start all services (PostgreSQL + App)
docker compose up -d

# Access the application
open http://localhost:8100

The Docker setup includes PostgreSQL 14 with pgvector pre-installed, all shards and UI bundled, automatic migrations, and no external dependencies (no Redis or Qdrant required).

Real Code Examples

Shard API Development Pattern

The README provides this pattern for building custom shard APIs:

from fastapi import APIRouter, Depends
from arkham_frame import get_frame

router = APIRouter(prefix="/api/myshard", tags=["myshard"])

@router.get("/items")
async def list_items(frame=Depends(get_frame)):
    # Access frame services
    db = frame.db
    events = frame.events
    llm = frame.llm  # Optional service
    return {"items": [...]}

This demonstrates the Frame's dependency injection pattern. Shards receive a frame object providing access to all 17 core services—database, event bus, LLM, vector store, etc.—without direct imports between shards. The llm service is optional; shards must handle its absence gracefully for air-gap or non-AI deployments.

Environment Configuration for Air-Gap Deployment

# .env for air-gapped deployment
DATABASE_URL=postgresql://user:pass@localhost:5432/shattered

# Enable offline mode (prevents model download attempts)
ARKHAM_OFFLINE_MODE=true

# Custom model cache location (if different from default)
ARKHAM_MODEL_CACHE=/path/to/huggingface/hub

# Local LLM endpoint
LLM_ENDPOINT=http://localhost:1234/v1

# Vision LLM for OCR (optional)
VLM_ENDPOINT=http://localhost:1234/v1

The ARKHAM_OFFLINE_MODE flag is critical for isolated networks—it prevents HuggingFace model download attempts. Embedding models must be pre-cached on a connected machine and copied to the air-gapped system at the specified cache path.

Production Deployment with Traefik

# 1. Configure environment
cp .env.example .env

# Edit .env and set:
# - AUTH_SECRET_KEY (generate a secure key)
# - DOMAIN=your-domain.com
# - ACME_EMAIL=admin@your-domain.com

# 2. Create certificate storage
mkdir -p traefik
touch traefik/acme.json
chmod 600 traefik/acme.json

# 3. Start with HTTPS
docker compose -f docker-compose.yml -f docker-compose.traefik.yml up -d

This configures automatic Let's Encrypt certificates, HTTP→HTTPS redirect, security headers (HSTS, CSP, X-Frame-Options), and modern TLS. The chmod 600 on acme.json is essential—Traefik requires restricted permissions for its certificate storage.

Air-Gap Network Verification

# Monitor network traffic (Linux)
ss -tuln | grep ESTAB

# Or use netstat
netstat -an | grep ESTABLISHED

# The only connections should be to:
# - localhost (PostgreSQL, LLM server)
# - Your local network (if applicable)

This verification step confirms no unexpected external connections. The platform's PostgreSQL-only architecture (using SKIP LOCKED for job queues rather than Redis) eliminates a common infrastructure dependency that would complicate air-gap deployment.

Advanced Usage & Best Practices

Shard Selection for Minimal Footprint

The 26 shards cover diverse functionality, but not all are required. For a focused OSINT workflow, consider: Dashboard, Projects, Ingest, Documents, Parse, Embed, Entities, Search, Graph, Timeline, and Export. Omit ACH, Credibility, and Media Forensics if your use case doesn't require structured hypothesis testing or image authenticity verification. The Frame auto-discovers installed shards, so selective installation reduces attack surface and resource consumption.

Embedding Model Pre-caching

For air-gap or bandwidth-constrained deployments, pre-cache embedding models on a connected system via the UI at Settings → ML Models, then copy ~/.cache/huggingface/hub to the target environment. The default all-MiniLM-L6-v2 is lightweight and adequate for most semantic search; larger models improve quality at latency cost.

Local LLM Selection

Schema Isolation for Multi-Tenant Safety

Each shard's PostgreSQL schema isolation isn't merely organizational—it enables clean backup/restore of individual functional areas and limits blast radius if a shard's data model changes. For production, consider separate database roles with schema-specific grants.

Geo View Limitation in Air-Gap

The Graph shard's Geo View fetches OpenStreetMap tiles externally. In isolated environments, avoid this tab (all other 9+ graph views function offline) or deploy a local tile server with offline map data.

Comparison with Alternatives

Trade-offs: Palantir offers superior scale and enterprise integration at substantial cost. Cloud NLP APIs provide easier initial setup but force data externalization and ongoing usage fees. Custom pipelines maximize flexibility but require implementing ACH, provenance tracking, and visualization from scratch—months of work that ArkhamMirror provides immediately. The 443-star community is smaller than major projects; self-support or contribution may be necessary for edge cases.

FAQ

Does mantisfury/ArkhamMirror require cloud AI services?

No. Local LLMs via LM Studio, Ollama, or vLLM work fully. Cloud providers are optional one-click alternatives.

What hardware is needed for GPU acceleration?

Optional. The platform degrades gracefully to CPU. GPU accelerates embedding generation and LLM inference but isn't required.

Is there a hosted/SaaS version?

No. The project is self-hosted only, consistent with its local-first, privacy-preserving design.

How does multi-tenancy work?

Built-in authentication with admin/analyst/viewer roles. First startup creates the initial tenant via setup wizard.

Can I add custom analysis shards?

Yes. The manifest schema and arkham-shard-ach reference implementation document the ArkhamShard interface. No direct shard imports—use the event bus.

What about document languages beyond English?

PaddleOCR supports multiple languages. spaCy's NER requires appropriate models; non-English support depends on spaCy model availability.

Is the 443-star count concerning for project longevity?

The project shows active maintenance (last commit January 2026) and substantial codebase (~217,000 lines). As MIT-licensed open source, it can be forked and continued independently if needed.

Conclusion

mantisfury/ArkhamMirror occupies a specific, valuable niche: structured investigative analysis with verifiable data sovereignty. It's not a general-purpose NLP toolkit or an enterprise intelligence platform—it's a purpose-built system for individuals and organizations that cannot risk exposing source materials to cloud infrastructure, yet need AI-assisted speed for document-heavy investigations.

The 26-shard architecture provides genuine modularity, not marketing segmentation. The PostgreSQL-only infrastructure eliminates operational complexity. The structured analytic techniques—ACH, contradiction detection, provenance tracking—reflect actual intelligence community methodologies rather than superficial AI wrapping.

Best suited for: investigative journalists handling sensitive sources, legal advocates building evidence cases, researchers in data-sensitive domains, and any organization requiring air-gap capable document intelligence. Less suited for: teams wanting fully managed SaaS, or those needing real-time collaborative editing without self-hosted infrastructure.

The project is actively maintained, well-documented, and free under MIT License. Explore the codebase, try the Docker quick-start, or examine the [INTERNAL_LINK: open-source document analysis tools] landscape to see where it fits your workflow.

Get started: https://github.com/mantisfury/ArkhamMirror