kotaemon: Your Next Favorite RAG-based Chat Tool

Introduction

In the world of document management and question answering (QA), developers and end users often find themselves in need of a clean, customizable, and powerful tool. Enter kotaemon, an open-source RAG-based tool that promises to revolutionize how we interact with documents. This article will dive deep into what kotaemon offers, how to get started, and why it's quickly becoming a favorite among developers and end users alike.

What is kotaemon?

kotaemon is an open-source tool designed to provide a clean and customizable interface for chatting with your documents using Retrieval-Augmented Generation (RAG). Created by Cinnamon, this project aims to serve both end users looking for a user-friendly QA experience and developers building their own RAG pipelines. With a focus on flexibility and usability, kotaemon stands out in the rapidly growing field of document interaction tools.

Background and Context

The rise of AI and machine learning has led to significant advancements in natural language processing (NLP). One of the most promising areas is Retrieval-Augmented Generation (RAG), which combines retrieval mechanisms with generative models to provide accurate and contextually relevant answers. kotaemon leverages this technology to create a seamless and intuitive user experience for document QA.

Why is kotaemon Trending Now?

kotaemon is gaining traction due to its clean UI, compatibility with various LLMs, and ease of installation. It offers a comprehensive framework for developers to build and customize their own RAG pipelines while providing end users with a user-friendly interface to interact with their documents.

Key Features

kotaemon comes packed with features that make it a standout tool in the RAG-based QA space:

• Clean & Minimalistic UI: User-friendly interface for RAG-based QA.
• Support for Various LLMs: Compatible with LLM API providers (OpenAI, AzureOpenAI, Cohere, etc.) and local LLMs (via ollama and llama-cpp-python).
• Easy Installation: Simple scripts to get you started quickly.
• Framework for RAG Pipelines: Tools to build your own RAG-based document QA pipeline.
• Customizable UI: See your RAG pipeline in action with the provided UI, built with Gradio.
• Gradio Theme: If you use Gradio for development, check out our theme here.
• Host your own document QA (RAG) web-UI: Support multi-user login, organize your files in private/public collections, collaborate and share your favorite chat with others.
• Organize your LLM & Embedding models: Support both local LLMs & popular API providers (OpenAI, Azure, Ollama, Groq).
• Hybrid RAG pipeline: Sane default RAG pipeline with hybrid (full-text & vector) retriever and re-ranking to ensure best retrieval quality.
• Multi-modal QA support: Perform Question Answering on multiple documents with figures and tables support. Support multi-modal document parsing (selectable options on UI).
• Advanced citations with document preview: By default the system will provide detailed citations to ensure the correctness of LLM answers. View your citations (incl. relevant score) directly in the in-browser PDF viewer with highlights. Warning when retrieval pipeline return low relevant articles.
• Support complex reasoning methods: Use question decomposition to answer your complex/multi-hop question. Support agent-based reasoning with ReAct, ReWOO and other agents.
• Configurable settings UI: You can adjust most important aspects of retrieval & generation process on the UI (incl. prompts).
• Extensible: Being built on Gradio, you are free to customize or add any UI elements as you like. Also, we aim to support multiple strategies for document indexing & retrieval. GraphRAG indexing pipeline is provided as an example.

Use Cases

kotaemon excels in several real-world scenarios:

End Users: Seamless Document Interaction

End users can leverage kotaemon's clean UI to interact with their documents effortlessly. Whether you're a student, a researcher, or a professional, kotaemon provides a straightforward way to ask questions and get accurate answers from your documents.

Developers: Customizable RAG Pipelines

For developers, kotaemon serves as a robust framework to build and customize RAG pipelines. With support for various LLMs and a customizable UI, developers can tailor the tool to fit their specific needs.

Multi-User Collaboration

Teams can use kotaemon to collaborate on document QA tasks. With support for multi-user login and file organization, kotaemon makes it easy to share and review documents.

Local and Private RAG

kotaemon supports local and private RAG setups, allowing users to maintain control over their data. This is particularly useful for organizations with sensitive information.

Advanced Reasoning and Citations

Complex reasoning methods and advanced citations make kotaemon a powerful tool for in-depth document analysis. Users can decompose questions and get detailed citations to ensure the accuracy of answers.

Step-by-Step Installation & Setup Guide

System Requirements

• Python >= 3.10
• Docker (optional, but recommended)
• Unstructured (if you want to process files other than .pdf, .html, .mhtml, and .xlsx documents)

With Docker (Recommended)

Using the Full Version

docker run \
-e GRADIO_SERVER_NAME=0.0.0.0 \
-e GRADIO_SERVER_PORT=7860 \
-v ./ktem_app_data:/app/ktem_app_data \
-p 7860:7860 -it --rm \
ghcr.io/cinnamon/kotaemon:main-full

Using the Full Version with Ollama

docker run <...> ghcr.io/cinnamon/kotaemon:main-ollama

Using the Lite Version

docker run <...> ghcr.io/cinnamon/kotaemon:main-lite

Accessing the WebUI

Once everything is set up correctly, you can access the WebUI at http://localhost:7860/.

Additional Configuration

You can specify the platform by passing --platform in the docker run command. For example:

docker run \
-e GRADIO_SERVER_NAME=0.0.0.0 \
-e GRADIO_SERVER_PORT=7860 \
-v ./ktem_app_data:/app/ktem_app_data \
-p 7860:7860 -it --rm \
--platform linux/arm64 \
ghcr.io/cinnamon/kotaemon:main-lite

REAL Code Examples from the Repository

Example 1: Setting Up Docker with Full Version

docker run \
-e GRADIO_SERVER_NAME=0.0.0.0 \
-e GRADIO_SERVER_PORT=7860 \
-v ./ktem_app_data:/app/ktem_app_data \
-p 7860:7860 -it --rm \
ghcr.io/cinnamon/kotaemon:main-full

This command sets up the full version of kotaemon with Docker, allowing for additional file type support.

Example 2: Using the Lite Version with Ollama

docker run \
-e GRADIO_SERVER_NAME=0.0.0.0 \
-e GRADIO_SERVER_PORT=7860 \
-v ./ktem_app_data:/app/ktem_app_data \
-p 7860:7860 -it --rm \
ghcr.io/cinnamon/kotaemon:main-ollama

This example demonstrates how to use the lite version with Ollama for local/private RAG.

Example 3: Customizing the Docker Platform

docker run \
-e GRADIO_SERVER_NAME=0.0.0.0 \
-e GRADIO_SERVER_PORT=7860 \
-v ./ktem_app_data:/app/ktem_app_data \
-p 7860:7860 -it --rm \
--platform linux/arm64 \
ghcr.io/cinnamon/kotaemon:main-lite

This command customizes the Docker platform to run on linux/arm64, suitable for newer Macs.

Advanced Usage & Best Practices

Pro Tips

• Customize the UI: Leverage Gradio's customization options to tailor the UI to your specific needs.
• Optimize for Performance: Use the full version for additional file type support, but be mindful of the larger Docker image size.
• Secure Your Data: Always ensure that your data is secure, especially when using local and private RAG setups.

Optimization Strategies

• Use Efficient Retrieval Methods: Optimize your RAG pipeline by using efficient retrieval methods to ensure fast and accurate results.
• Regular Updates: Keep your kotaemon installation up to date to benefit from the latest features and bug fixes.

Comparison with Alternatives

When choosing a RAG-based tool, it's essential to compare kotaemon with other available options. Here's a comparison table to help you make an informed decision:

FAQ

Q1: How do I install kotaemon?

A: You can install kotaemon using Docker. Follow the installation guide provided in the User Guide.

Q2: What are the system requirements for kotaemon?

A: You need Python >= 3.10 and Docker (optional but recommended). Check the User Guide for detailed requirements.

Q3: Can I customize the UI?

A: Yes, kotaemon is built with Gradio, which allows for extensive customization of the UI.

Q4: How do I update kotaemon?

A: You can update kotaemon by downloading the latest release from the GitHub repository.

Q5: Is kotaemon secure?

A: kotaemon supports local and private RAG setups, ensuring you maintain control over your data.

Q6: Can I use kotaemon for commercial purposes?

A: Yes, kotaemon is open-source and can be used for commercial purposes.

Q7: How can I get support for kotaemon?

A: You can get support by opening an issue on the GitHub repository or contacting the maintainers via email.

Conclusion

kotaemon is a powerful and flexible RAG-based tool that transforms the way we interact with documents. With its clean UI, support for various LLMs, and customizable features, kotaemon is a must-have for developers and end users alike. Whether you're looking to build your own RAG pipeline or simply need a user-friendly QA tool, kotaemon has you covered. Check out the GitHub repository to get started today!