AI Town: Build Your Own Autonomous AI Agent Society

The future of artificial intelligence isn't just about chatbots—it's about entire societies of autonomous agents that live, breathe, and interact in persistent virtual worlds. But building these complex simulations has been a nightmare of infrastructure challenges, fragmented tools, and Python-only frameworks. Enter AI Town, the revolutionary TypeScript starter kit that's democratizing autonomous agent development for JavaScript developers everywhere.

If you've been fascinated by research papers showing AI characters forming relationships, planning parties, and exhibiting emergent behaviors, but felt locked out because you don't speak Python, your moment has arrived. AI Town, created by a16z-infra, transforms the cutting-edge "Generative Agents" research into a deployable, customizable playground that runs on modern web technologies. This isn't just a toy project—it's a production-ready foundation for everything from AI research platforms to multiplayer games with truly intelligent NPCs.

In this deep dive, we'll explore how AI Town eliminates infrastructure headaches, examine its powerful Convex backend architecture, walk through real code examples, and show you exactly how to launch your own virtual society in minutes. Whether you're a researcher studying emergent behaviors, a game developer building the next generation of NPCs, or a curious developer wanting to experiment with autonomous agents, this guide will equip you with everything you need to start building immediately.

What is AI Town?

AI Town is a MIT-licensed, deployable starter kit for building and customizing your own version of AI Town—a virtual town where AI characters live, chat, and socialize autonomously. Born from the legendary research paper Generative Agents: Interactive Simulacra of Human Behavior, this project represents a seismic shift in how developers can experiment with and deploy autonomous agent simulations.

The brainchild of a16z-infra (Andreessen Horowitz Infrastructure team), AI Town addresses a critical gap in the AI development ecosystem. While the original research implementation and most simulators in this space are locked in Python, AI Town brings these capabilities to the JavaScript/TypeScript world, opening the door for millions of web developers to participate in autonomous agent development.

At its core, AI Town is a full-stack application that combines a powerful game engine, vector database, real-time synchronization, and LLM integration into a single cohesive package. The project leverages Convex as its backend powerhouse, handling everything from shared global state and transactions to simulation engine logic and vector search. This isn't just a wrapper around existing tools—it's a complete platform designed for serious development and scalability.

What makes AI Town genuinely revolutionary is its dual deployment model. You can run everything locally using Ollama for complete data privacy and zero API costs, or scale to the cloud using providers like OpenAI and Together.ai. This flexibility makes it equally suitable for hobbyists experimenting on their laptops and enterprises building production-grade simulations. The inclusion of PixiJS for rendering creates a rich, interactive 2D world where you can actually watch your agents navigate, interact, and build relationships in real-time.

The project has exploded in popularity because it arrives at the perfect moment—when interest in autonomous agents is at an all-time high, but accessible tools remain scarce. By providing a TypeScript foundation with modern deployment options including Docker and Fly.io, AI Town has become the go-to starter kit for anyone serious about building the next generation of AI-powered virtual worlds.

Key Features That Make AI Town Essential

AI Town's architecture is a masterclass in modern full-stack development, combining cutting-edge technologies into a seamless development experience. Let's dissect the powerhouse features that make this framework indispensable for autonomous agent development.

Convex Backend: The Simulation Engine Powerhouse

The heart of AI Town beats with Convex, a reactive database and application platform that eliminates traditional backend complexity. Unlike conventional setups requiring separate database, API layer, and real-time sync services, Convex provides shared global state, ACID transactions, and a built-in simulation engine out of the box. This means your agents can read and write to a consistent world state without you writing a single line of API code. The vector search capability enables semantic memory retrieval, allowing agents to recall relevant past experiences naturally—critical for believable emergent behavior.

Universal LLM Integration: Local to Cloud

AI Town shatters the vendor lock-in problem with its configurable LLM architecture. By default, it uses Ollama to run Llama3 entirely on your machine, giving you complete control over data privacy and costs. But with a simple configuration change, you can switch to OpenAI, Together.ai, or any OpenAI-compatible API. The framework handles model-specific quirks automatically, letting you focus on agent logic rather than API integrations. This flexibility is unprecedented in the autonomous agent space.

PixiJS Rendering: Watch Intelligence Emerge

Theory becomes reality through PixiJS, a blazing-fast 2D WebGL renderer that brings your agent society to life. Every conversation, movement, and social interaction renders in real-time in your browser. The included tilesets and pixel art create an immersive retro aesthetic, while the rendering system is fully extensible for custom graphics. This visual feedback loop is invaluable for debugging agent behaviors and showcasing your simulation to stakeholders.

Modern Deployment Ecosystem

AI Town embraces contemporary DevOps practices with multiple deployment pathways. The standard Convex setup offers one-click cloud deployment with hot-reloading. Docker Compose provides self-hosted backend capabilities for air-gapped environments. Fly.io configuration enables edge deployment for low-latency global access. A community Pinokio fork even offers one-click installation for non-developers. This versatility ensures AI Town fits any infrastructure requirement.

Authentication & Security

Optional Clerk integration adds production-ready authentication in minutes, not hours. This is crucial for multiplayer scenarios where you want users to have accounts, save progress, or manage their own agent populations. The security model extends to API key management for LLM providers, keeping secrets out of your codebase.

Generative Content Pipeline

Beyond agent conversations, AI Town includes Replicate integration for MusicGen, automatically generating background music that fits your world's atmosphere. Pixel art generation through Replicate and Fal.ai means you can procedurally generate character sprites and environmental assets, creating infinite variety without manual art creation.

Real-World Use Cases That Transform Development

AI Town isn't a theoretical playground—it's a practical tool solving concrete problems across multiple domains. Here are five powerful scenarios where this framework shines.

1. Academic Research in Emergent Behavior

Researchers studying how AI agents form societies, develop cultures, or solve collective action problems need a reproducible platform. AI Town provides version-controlled agent definitions, deterministic simulation parameters, and detailed logging of every interaction. Unlike the original Python implementation, you can deploy your study online, allowing remote participants to observe and interact with simulations in real-time. The TypeScript codebase is also more accessible to web-native research assistants.

2. Game Development with Truly Intelligent NPCs

Traditional game NPCs follow rigid behavior trees that break immersion. AI Town enables persistent NPCs with memory, relationships, and goals that evolve across play sessions. Imagine a village in an RPG where shopkeepers remember your haggling tactics, guards gossip about your exploits, and characters form friendships that affect quest availability. The Convex backend ensures state persists even when players log off, creating a living world that continues without them.

3. Social Dynamics Training Simulations

Corporations and governments need to model how information spreads, how groups make decisions, or how interventions affect community behavior. AI Town's configurable agent personalities and communication patterns let you create microcosms of real societies. Test crisis communication strategies, simulate market dynamics, or model viral marketing campaigns with agents that exhibit human-like social behaviors. The visual interface makes results presentable to non-technical stakeholders.

4. Interactive Storytelling and Entertainment

Content creators can build persistent narrative worlds where AI characters live out stories that audiences can influence. Think Twitch Plays Pokémon meets reality TV, but with AI contestants. Each agent has desires, fears, and relationships that create emergent drama. The Clerk authentication lets fans create accounts, vote on world events, or even inject their own agents into the simulation, creating a new form of interactive entertainment.

5. AI Safety and Alignment Testing

Before deploying AI systems in the real world, we need safe testing environments. AI Town provides a sandboxed society where you can observe how agents pursue goals, whether they develop deceptive behaviors, and how they respond to value alignment interventions. The ability to run thousands of simulations locally with Ollama makes large-scale safety testing affordable and accessible to smaller research labs.

Step-by-Step Installation & Setup Guide

Getting AI Town running is straightforward, but the multiple deployment options require careful attention to detail. Follow these comprehensive steps to launch your autonomous agent society.

Prerequisites

Before starting, ensure you have:

• Node.js 18+ and npm installed
• Git for cloning the repository
• A Convex account (free at convex.dev) for cloud deployment
• For local inference: Ollama installed (optional but recommended)

Standard Local Development Setup

This is the fastest way to get started with full hot-reloading and cloud backend sync.

# Clone the repository from GitHub
git clone https://github.com/a16z-infra/ai-town.git

# Navigate into the project directory
cd ai-town

# Install all dependencies (this will prompt for Convex login)
npm install

The installation process automatically sets up the Convex CLI and links your project to your Convex account. This creates a new deployment target in the cloud where your backend functions will live.

# Launch the full development environment
npm run dev

This single command starts both frontend and backend development servers with hot module replacement. Your browser will automatically open to http://localhost:5173 where you'll see your AI Town come to life.

For separate terminal control (useful for debugging):

# Terminal 1: Frontend development
npm run dev:frontend

# Terminal 2: Backend development with live sync
npm run dev:backend

The backend terminal continuously syncs your Convex functions to the cloud as you save, while the frontend terminal provides instant UI updates.

Docker Compose Self-Hosted Setup

For complete control or air-gapped environments, run everything locally with Docker.

# Build and start all services (backend, frontend, dashboard)
docker compose up --build -d

The -d flag runs containers in detached mode. After the initial build, you can manage services with:

docker compose stop  # Stop services
docker compose start # Restart services

Access Points:

• Frontend: http://localhost:5173
• Backend API: http://localhost:3210
• Convex Dashboard: http://localhost:6791

Critical Configuration Step: Generate an admin key for dashboard access:

docker compose exec backend ./generate_admin_key.sh

Copy the generated key and create a .env.local file:

# .env.local
CONVEX_SELF_HOSTED_ADMIN_KEY="your-generated-admin-key-here"
CONVEX_SELF_HOSTED_URL="http://127.0.0.1:3210"

Warning: If you run docker compose down, you'll need to regenerate the key and update .env.local.

Initialize the Convex backend schema:

npm run predev

Then deploy functions continuously:

npm run dev:backend

Ollama Configuration for Local Inference

AI Town defaults to Ollama for free, private AI inference. After installing Ollama:

# Pull the default chat model
ollama pull llama3

# Test the model
ollama run llama3

If using Docker, configure it to reach your host Ollama:

# Tell Convex backend where to find Ollama
npx convex env set OLLAMA_HOST http://host.docker.internal:11434

# Test connectivity from within Docker
docker compose exec backend /bin/bash curl http://host.docker.internal:11434

You should see "Ollama is running" if configured correctly.

REAL Code Examples from the Repository

Let's examine actual code snippets from AI Town's README, breaking down exactly how to configure and customize your autonomous agent society.

Example 1: Standard Development Commands

These are the fundamental commands you'll use daily:

# Clone and enter the project
git clone https://github.com/a16z-infra/ai-town.git
cd ai-town

# Install dependencies - this handles Convex CLI setup automatically
npm install

# Launch full development environment
npm run dev

Explanation: The npm install command is particularly powerful here—it doesn't just install packages; it bootstraps your Convex deployment, linking your local project to a cloud backend. The npm run dev script is a convenience wrapper that concurrently runs both frontend and backend development servers, enabling hot-reloading across the entire stack. This is defined in the project's package.json and represents the fastest path from zero to running simulation.

Example 2: Docker Compose Self-Hosted Deployment

For production-like environments or complete data control:

# Build and start all services in detached mode
docker compose up --build -d

# Generate admin key for dashboard access (CRITICAL SECURITY STEP)
docker compose exec backend ./generate_admin_key.sh

Explanation: The Docker Compose configuration orchestrates three services: the Next.js frontend, the Convex backend, and the Convex dashboard. The --build flag ensures you're building fresh images with your local code changes. The admin key generation is a one-time (per container lifecycle) operation that creates a secret token for authenticating with the self-hosted Convex dashboard. Without this, you cannot deploy schema changes or view backend logs.

Example 3: Ollama Model Configuration

Switching AI models is straightforward:

# Pull the default Llama3 model
ollama pull llama3

# Customize which model to use (optional)
npx convex env set OLLAMA_MODEL llama3:70b  # Use larger model

# Configure embedding model for memory
npx convex env set OLLAMA_EMBEDDING_MODEL mxbai-embed-large

Explanation: AI Town uses two separate models: a chat model for agent conversations and an embedding model for semantic memory search. The npx convex env set commands persist these settings to your Convex environment, making them available to backend functions at runtime. This separation allows you to optimize each component independently—using a smaller, faster model for chat and a specialized model for embeddings.

Example 4: OpenAI Integration Configuration

To switch from local Ollama to cloud OpenAI:

// In convex/util/llm.ts change the following line:
export const EMBEDDING_DIMENSION = OPENAI_EMBEDDING_DIMENSION;

// Then set your API key
npx convex env set OPENAI_API_KEY 'sk-your-secret-key-here'

Explanation: This code snippet shows the elegant abstraction layer in AI Town. By simply changing which EMBEDDING_DIMENSION constant is exported, you switch the entire backend from Ollama's embedding size to OpenAI's 1536 dimensions. The rest of the codebase remains unchanged. The npx convex env set command securely stores your API key in Convex's encrypted environment variables, never exposing it in client-side code.

Example 5: Together.ai Provider Setup

For alternative cloud providers:

// In convex/util/llm.ts change the following line:
export const EMBEDDING_DIMENSION = TOGETHER_EMBEDDING_DIMENSION;

// Set your Together.ai API key
npx convex env set TOGETHER_API_KEY 'your-together-key'

Explanation: Together.ai offers access to numerous open-source models through a single API. The configuration pattern mirrors OpenAI's, demonstrating AI Town's provider-agnostic design. The TOGETHER_EMBEDDING_DIMENSION constant is pre-configured for their default embedding model, ensuring seamless switching. This modular approach means you can benchmark different providers against each other with minimal code changes.

Example 6: Docker Environment Configuration

Critical for self-hosted deployments:

# .env.local
CONVEX_SELF_HOSTED_ADMIN_KEY="admin-key-from-generate_admin_key.sh"
CONVEX_SELF_HOSTED_URL="http://127.0.0.1:3210"

Explanation: These environment variables tell the Convex CLI how to connect to your self-hosted backend instead of the cloud service. The CONVEX_SELF_HOSTED_URL points to the Docker container's exposed port, while the admin key authenticates you as a deployment administrator. The quotes around the admin key are crucial—special characters in the token can break the configuration without proper quoting.

Advanced Usage & Best Practices

Once you've mastered the basics, these pro tips will elevate your AI Town from demo to production-ready system.

Optimize Memory Search Performance

Agent memory is the key to believable behavior, but searching large memory stores can slow responses. In convex/constants.ts, adjust:

export const NUM_MEMORIES_TO_SEARCH = 3; // Reduced from default 5-10

This limits the conversation context, significantly speeding up LLM calls while maintaining personality consistency. For research simulations, increase this value to 10-15 for richer historical context.

Customize Agent Personalities

Don't settle for generic agents. In the convex/agents directory, modify the personality matrices:

// Add custom traits, relationships, and initial memories
const customAgent = {
  name: "Dr. Elena",
  personality: {
    openness: 0.9,
    conscientiousness: 0.7,
    extraversion: 0.4,
    agreeableness: 0.8,
    neuroticism: 0.2
  },
  initialMemories: [
    "You are a quantum physicist who distrusts simple explanations",
    "You had a heated debate with Marcus about multiverse theory last week"
  ]
};

Scale with Convex Scheduling

For simulations with 50+ agents, use Convex's built-in scheduling to prevent function timeouts:

// In your agent update function
await ctx.scheduler.runAfter(100, internal.agents.tick, { agentId });

This distributes agent updates across time, preventing burst load on your backend.

Implement Custom LLM Providers

Want to use Anthropic Claude or Google Gemini? The abstraction layer in convex/util/llm.ts makes this straightforward. Create a new provider file, implement the chatCompletion interface, and add your embedding dimension constant. The modular design welcomes pull requests for new providers.

Production Deployment Checklist

Before launching publicly:

1. Enable Clerk authentication to prevent abuse
2. Set rate limits on agent creation via Convex HTTP actions
3. Monitor costs with provider-specific usage dashboards
4. Backup agent states using Convex's export functionality
5. Configure CORS properly if embedding in external sites

Comparison with Alternatives

Why choose AI Town over other agent simulation frameworks? The differences are stark.

Key Differentiators:

• JavaScript Ecosystem: AI Town is the only serious autonomous agent framework for TypeScript developers, leveraging npm's vast library ecosystem.

• Zero Backend Code: Convex eliminates the need to write API endpoints, WebSocket handlers, or database queries—saving weeks of development time.

• True Local-First: Ollama integration means you can develop and deploy without any cloud dependencies, crucial for privacy-sensitive applications.

• Web-Native: The PixiJS renderer works everywhere the web does—no plugins, no installs, just share a URL.

Frequently Asked Questions

Q: What exactly is AI Town and how is it different from a chatbot?

A: AI Town is a simulation platform where multiple AI agents interact autonomously in a persistent world. Unlike single-turn chatbots, agents have memories, form relationships, and continue living even when you're not watching—creating emergent social dynamics.

Q: How much does it cost to run AI Town?

A: Running locally with Ollama is completely free after initial hardware investment. Cloud deployment costs depend on your scale: Convex has a generous free tier, and LLM API costs vary by provider. A small town of 10 agents might cost $5-20/month on cloud providers.

Q: Can I use my own fine-tuned models?

A: Absolutely! With Ollama, you can run any GGUF model locally. For cloud providers, any OpenAI-compatible endpoint works. The modular LLM interface makes adding custom providers straightforward.

Q: Is AI Town production-ready?

A: Yes. The MIT license permits commercial use, and the stack (Convex, Clerk, PixiJS) powers production applications worldwide. Many developers have already launched AI Town-based projects to users.

Q: How many agents can the system handle?

A: On a modest server, 50-100 concurrent agents perform well. The Convex backend scales horizontally, so you can support thousands by optimizing update frequencies and using scheduled functions. Local Ollama deployments are limited by your GPU memory.

Q: What's the difference between local and cloud deployment?

A: Local deployment via Docker gives you complete data privacy and control but requires managing infrastructure. Cloud deployment with Convex offers automatic scaling, global CDN, and zero server maintenance at the cost of small latency increases and potential API fees.

Q: How do I customize agent appearances and the world map?

A: Agent sprites are in the public/assets/characters directory—replace PNG files to change appearances. The world map is defined in convex/world.ts as a tile matrix. Use the Tiled editor to create new maps and export them to the expected JSON format.

Conclusion: Your Autonomous Agent Journey Starts Now

AI Town represents more than just a clever demo—it's a fundamental shift in how we build and deploy autonomous agent systems. By packaging complex research into an accessible TypeScript framework, a16z-infra has democratized a technology that was previously confined to academic labs and Python specialists.

The combination of Convex's reactive backend, Ollama's local inference, and PixiJS's rich rendering creates a development experience that is both powerful and joyful. You can literally go from reading the research paper to running your own agent society in under ten minutes, then deploy it to the cloud for the world to see.

What excites me most is the extensibility. The clean abstraction layers invite experimentation—swap LLMs, add new agent senses, integrate external data sources, or build entirely new interaction modalities. The MIT license means you're free to build commercial products, research platforms, or artistic installations without restriction.

The autonomous agent revolution isn't coming—it's here. And with AI Town, you're holding the keys to participate fully. Whether you're building the next viral social simulation, conducting groundbreaking research, or simply exploring the future of AI, this starter kit provides the foundation you need.

Ready to build your own society? Clone the repository, run npm install, and watch your first agents come to life. The future is waiting at github.com/a16z-infra/ai-town.