kreuzberg-dev/kreuzberg: Polyglot Document Intelligence with a Rust Core

kreuzberg-dev/kreuzberg: Polyglot Document Intelligence with a Rust Core

Developers building document pipelines face a familiar mess: one library for PDFs, another for Office files, a third for OCR, then glue code to stitch them together. Each dependency brings its own failures, version conflicts, and deployment headaches. For teams processing documents at scale—whether for search indexes, RAG pipelines, or compliance workflows—this fragmentation burns engineering time and introduces reliability gaps.

kreuzberg-dev/kreuzberg addresses this directly. It's a polyglot document intelligence framework built on a Rust core, designed to extract text, metadata, images, and structured information from 97+ formats through a single engine. With native bindings for 15 languages and deployment options spanning library, CLI, REST API, and MCP server, it eliminates the need to assemble and maintain multiple extraction tools. The project has accumulated 8,643 GitHub stars and 518 forks, with active development as of July 2026.

What is kreuzberg-dev/kreuzberg?

kreuzberg-dev/kreuzberg is an open-source document processing framework maintained under the MIT license. Its architecture centers on a Rust core that handles format detection, reading, and extraction, wrapped in language bindings that let developers call it from virtually any mainstream stack.

The project's scope extends well beyond simple text extraction. It processes PDFs, Office documents, images, audio, video, archives, email, scientific publications, and code files—applying intelligent MIME detection and streaming large files without loading them entirely into memory. Output formats include plain text, Markdown^{↗ Smart Converter}, Djot, HTML, JSON tree structures, and structured JSON with OCR metadata and bounding boxes.

What distinguishes kreuzberg-dev/kreuzberg in the current landscape is its combination of breadth and deployment flexibility. Most extraction tools specialize: some handle PDFs well but choke on Office files; others offer OCR but require GPU infrastructure. Kreuzberg runs CPU-only, supports batch parallelization with configurable concurrency, and caches results by content hash to avoid redundant processing. The 15 language bindings—Rust, Python^{↗ Bright Coding Blog}, Node.js/TypeScript (including Bun, Wasm, Deno), Java, Go, C#, PHP^{↗ Bright Coding Blog}, Ruby, Elixir, Dart, Swift, Kotlin, Zig, and C via FFI—reflect a deliberate investment in accessibility across technology ecosystems.

The framework also integrates with modern AI workflows through its MCP server implementation, exposing 9 tools, 3 prompts, and 4 resources to Claude, Cursor, and similar agent interfaces. This positions it as infrastructure for document-aware AI applications, not merely a standalone utility.

Key Features

Unified format coverage. The engine handles 97 file formats across 8 categories: Office documents (Word, Excel, PowerPoint, ODF, Pages, Numbers, Keynote), PDFs, images (including HEIC, AVIF, JPEG 2000, and JBIG2), audio and video (via Whisper ONNX transcription), web markup, structured data, email and archives, and scientific formats (LaTeX, Typst, Jupyter notebooks, JATS, citation formats).

Multiple OCR backends with fallback chains. Tesseract (native C FFI and WASM), PaddleOCR (ONNX Runtime, mobile-optimized), Candle (pure Rust, CPU-only), and VLM options (GPT-4 Vision, Claude Vision, Gemini Vision, or 143 providers via liter-llm integration). The system falls back through configured backends automatically and reports confidence scores.

Code intelligence for 306 languages. Via tree-sitter, kreuzberg-dev/kreuzberg extracts functions, classes, imports, symbols, and docstrings, with syntax-aware chunking designed specifically for RAG pipelines. This targets a genuine gap: most document processors treat code as plain text, losing structural information that matters for semantic search.

Six output formats. Plain text for simplicity, Markdown and Djot for readability, HTML for browser rendering, JSON tree for machine processing, and Structured JSON with OCR metadata including bounding boxes and confidence scores.

Deployment modes. Library integration for application embedding, CLI with 12 commands, REST API server (xberg serve), MCP server for AI agent integration, and Docker^{↗ Bright Coding Blog} containers with Helm charts for Kubernetes.

Performance features. Streaming for multi-GB files, parallel batch processing with per-file timeouts, content-hash caching, and zero GPU requirement for core operations.

Enrichment pipeline. NER, PII redaction, summarization, translation, QR code detection, page classification, keyword extraction (YAKE/RAKE), language detection, layout detection, table extraction, and token reduction via TOON wire format.

Use Cases

Enterprise document search and compliance. Organizations with decades of file archives in heterogeneous formats—PDFs from scanned records, legacy Word documents, email exports—can normalize everything to structured text without maintaining separate extraction pipelines per format. The caching layer and streaming support make this practical for terabyte-scale backlogs.

RAG pipeline ingestion. The code intelligence features (306 languages, syntax-aware chunking) and structured output formats directly support retrieval-augmented generation systems. Developers can feed documents, codebases, and technical specifications into vector stores with semantic boundaries preserved, rather than splitting arbitrarily on character counts.

AI agent document interaction. The MCP server exposes extraction capabilities to Claude, Cursor, and compatible agents. A developer working in an AI-assisted IDE can point at a PDF or spreadsheet and receive structured content without leaving their workflow, with the agent understanding document structure through the MCP resource definitions.

Media transcription pipelines. Audio and video files (MP3, M4A, WAV, WebM, MP4) yield transcripts via Whisper ONNX without requiring separate speech-to-text infrastructure. This suits podcast indexing, meeting archive search, and content moderation workflows.

Automated PII handling. The redaction and NER capabilities support GDPR/CCPA compliance pipelines, identifying and masking phone numbers, emails, SSNs, credit cards, and addresses during document processing.

Installation & Setup

Python

pip install xberg

The Python package provides async extraction APIs. See the Python README for full documentation.

Node.js / TypeScript

npm install @xberg-io/xberg

For WebAssembly environments:

npm install @xberg-io/xberg-wasm

Rust

cargo add xberg

Go

go get github.com/xberg-io/xberg

Java

Available on Maven Central as io.xberg:xberg. See the Java README for the exact dependency snippet.

C#

dotnet add package Xberg

Ruby

gem install xberg

PHP

composer require xberg-io/xberg

Elixir

Add {:xberg, "~> 1.0"} to your mix.exs dependencies.

CLI via Homebrew

brew install xberg-io/tap/xberg

The CLI provides 12 commands: extract, batch, detect, formats, version, cache (with stats, clear, manifest, warm subcommands), serve, mcp, api, embed, chunk, and completions.

Docker

docker pull ghcr.io/xberg-io/xberg:latest

The container supports API, CLI, and MCP modes. See the Docker guide for configuration examples.

REST API Server

xberg serve --host 0.0.0.0 --port 8000

Single POST endpoint handles all formats, returns JSON or Markdown, and streams large files.

MCP Server

xberg mcp --transport stdio

Configure in Claude Desktop or Cursor:

{
  "mcpServers": {
    "xberg": { "command": "xberg", "args": ["mcp"] }
  }
}

Real Code Examples

The README provides one explicit Rust example for basic extraction:

use xberg::{extract, ExtractInput, ExtractionConfig};

#[tokio::main]
async fn main() -> xberg::Result<()> {
    let config = ExtractionConfig::default();
    let output = extract(
        ExtractInput::from_uri("document.pdf"),
        &config
    ).await?;

    println!("{}", output.results[0].content);
    Ok(())
}

This demonstrates the core pattern: construct a default configuration, create an ExtractInput from a URI (local path or URL), await the async extract call, and access results through the output structure. The ExtractInput::from_uri method handles format detection automatically—no manual MIME type specification needed.

The ExtractionConfig::default() provides sensible baseline settings. For production use, you would customize OCR backend selection, output format, caching behavior, and timeout parameters through this configuration object. The async design allows integration into Tokio-based applications without blocking the runtime.

Note that the README currently documents this single code example prominently. Developers working in other languages should consult the per-binding READMEs linked in the installation section, as the API patterns vary by language idioms—Python uses async/await similarly, while Go and Java expose their own concurrency models. The relative scarcity of inline examples in the main README suggests the project expects users to reference language-specific documentation rather than centralizing all examples.

Advanced Usage & Best Practices

Configure OCR fallback chains deliberately. The README notes multiple OCR backends but doesn't prescribe specific fallback ordering. For production deployments, test Tesseract first for scanned documents (mature, widely deployed), PaddleOCR for mobile-originated images (optimized for scene text), and Candle when minimizing dependencies matters. VLM backends add latency and cost—reserve them for complex layouts where traditional OCR fails.

Leverage content-hash caching for batch workflows. The cache uses file content plus configuration as its key, not file path. This means renaming or moving files won't trigger re-extraction, but changing extraction parameters will. Design batch jobs with stable configurations to maximize cache hits.

Use streaming for archives and large files. The framework supports multi-GB files, but memory pressure depends on enabling streaming modes. For nested archives (zip within zip), the crawl mode recursively extracts documents-within-documents—enable this explicitly, as it changes security and resource boundaries.

Match output format to downstream consumption. Plain text minimizes token counts for LLM prompts. Markdown preserves structure for human review. Structured JSON with bounding boxes enables precise source attribution in RAG systems—linking generated answers back to specific document regions.

Monitor batch concurrency. The max_concurrent_extractions parameter controls parallelism. Set this based on your I/O bandwidth and OCR backend capacity; CPU-bound backends like Tesseract benefit from concurrency matching core count, while remote VLM calls require connection pool management.

Comparison with Alternatives

kreuzberg-dev/kreuzberg occupies a distinct position: broader language binding coverage than Tika or Unstructured, fully open-source and self-hostable unlike LlamaParse, with code intelligence features absent from all three. The trade-off is ecosystem maturity—Tika has years of production deployment history, while kreuzberg-dev/kreuzberg's v1 line (as Xberg) represents a recent rebuild. Teams prioritizing polyglot integration and modern AI agent interfaces may find the trade-off favorable; those needing battle-tested Java ecosystem integration may prefer Tika pending broader production validation.

FAQ

What license covers kreuzberg-dev/kreuzberg? MIT License. Commercial use, modification, and distribution are permitted.

Does it require a GPU? No. Core extraction runs CPU-only. GPU accelerates optional VLM OCR backends but isn't required.

Can I process documents without installing language-specific packages? Yes. Use the CLI, Docker container, or REST API server for language-agnostic deployment.

How does the MCP server differ from the REST API? MCP exposes tools, prompts, and resources to AI agents (Claude, Cursor) via stdio or HTTP. REST API provides traditional HTTP endpoints for programmatic integration.

Is there a hosted/cloud version? The README mentions Xberg Enterprise as a separate managed extraction API with dashboards and observability, distinct from this open-source core.

What happened to the Kreuzberg name? Xberg is the next iteration of Kreuzberg v4 LTS—same engine, rebuilt and rebranded under a fresh v1 line.

How current is development? Last commit dated 2026-07-14 per repository statistics, indicating active maintenance.

Conclusion

kreuzberg-dev/kreuzberg targets a genuine infrastructure gap: unified document processing without the typical assembly of format-specific tools. The Rust core delivers performance and deployment flexibility, while 15 language bindings and multiple deployment modes (library, CLI, API, MCP, Docker) accommodate diverse engineering contexts.

It's best suited for teams building document pipelines where format diversity, code intelligence, and AI agent integration matter more than single-format optimization. The 8,643-star traction suggests resonance with developers tired of maintaining extraction glue code. Those needing only PDF text extraction may find lighter tools sufficient; those processing heterogeneous document corpora at scale, or feeding structured content into RAG and agent systems, should evaluate it seriously.

Explore the repository, try the live demo, or install via your preferred package manager: https://github.com/kreuzberg-dev/kreuzberg

For related document processing strategies, see [INTERNAL_LINK: building-scalable-rag-pipelines] and [INTERNAL_LINK: rust-tools-for-ml-infrastructure].