Local AI inference engine and agent platform for image generation, speech processing, and tool-based automation using LLMs
AI Runner: Offline AI Inference Engine for Hackers, Makers, and Builders.
Here are some of the things you can do with AI Runner:
- ✅ Voice-based chatbots for real-time conversations (three text-to-speech engines each with multiple voices to choose from)
- ✅ Customizable agents with names, moods and personalities for more engaging conversations
- ✅ Retrieval-Augmented Generation (RAG) for documents and websites
- ✅ Text-to-Image generation with Stable Diffusion and ControlNet
- ✅ Image manipulation with inpainting, outpainting, ControlNet and Image filters
For extra security, performance, and compatibility, AI Runner is built with Wayland support, Python 3.13, and the latest stable torch libraries.
System Requirements
| Specification | Minimum | Recommended |
|---|---|---|
| OS | Ubuntu 22.04, Windows 10 | Ubuntu 22.04 (Wayland) |
| CPU | Ryzen 2700K or Intel Core i7-8700K | Ryzen 5800X or Intel Core i7-11700K |
| Memory | 16 GB RAM | 32 GB RAM |
| GPU | NVIDIA RTX 3060 or better | NVIDIA RTX 4090 or better |
| Network | Broadband (used to download models) | Broadband (used to download models) |
| Storage | 22 GB | 50 GB |
💾 Installation Quick Start
🔧 Installation Steps
- Install system requirements
sudo apt update && sudo apt upgrade -y sudo apt install -y make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev python3-openssl git nvidia-cuda-toolkit pipewire libportaudio2 libxcb-cursor0 gnupg gpg-agent pinentry-curses espeak xclip cmake qt6-qpa-plugins qt6-wayland qt6-gtk-platformtheme mecab libmecab-dev mecab-ipadic-utf8 sudo apt install espeak sudo apt install espeak-ng-espeak - Create
airunnerdirectorysudo mkdir ~/.local/share/airunner sudo chown $USER:$USER ~/.local/share/airunner - Install AI Runner - Python 3.13+ required
pyenvandvenvare recommended (see wiki for more info)pip install "typing-extensions==4.13.2" pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128 pip install airunner[all_dev] pip install -U timm - Run AI Runner
airunner
For more options, including Docker, see the Installation Wiki.
Note: AI Runner does not distribute AI art models. You are responsible for obtaining and your own.
🛠️ Usage
Basic Usage
- Run AI Runner:
airunner - Build templates:
airunner-build-ui
⭐ Features
Below is a high-level list of capabilities in AI Runner:
| Feature | Description |
|---|---|
| LLMs & Communication | |
| Voice-based chatbot conversations | Have real-time voice-chat sessions with an LLM (speech-to-text + text-to-speech) |
| Text-to-speech (TTS) | Convert text to spoken audio using OpenVoice, SpeechT5, and Espeak |
| Speech-to-text (STT) | Convert spoken audio to text with Whisper |
| Customizable chatbots | Create AI personalities and moods for more engaging conversations |
| Retrieval-Augmented Generation | Use local doc or website data to enrich chat responses |
| Image Generation | |
| Stable Diffusion (1.5, SDXL, Turbo) | Generate images from textual prompts, sketches, or existing images |
| Drawing tools & ControlNet | Fine-tune image outputs with extra input or guides |
| LoRA & Embeddings | Load LoRA models or textual embeddings for specialized image generation |
| Image Manipulation | |
| Inpaint & Outpaint | Modify portions of generated images while keeping context |
| Image filters | Blur, film grain, pixel art, etc. |
| Utility | |
| Offline | Everything runs locally, no external API required |
| Fast generation | E.g., ~2 seconds on an RTX 2080s for stable diffusion |
| Docker-based approach | Simplifies setup & ensures GPU acceleration works out of the box |
| Dark mode | Built-in theming (Light / Dark / System) |
| NSFW toggles | Enable or disable NSFW detection for images |
| Ethical guardrails | Basic guardrails for safe LLM usage (optional) |
| Extensions | Build your own feature add-ons via the extension API |
| Python Library | pip install airunner and embed it in your own projects |
| API Support | Optionally use OpenRouter or other external LLMs |
⚙️ System Requirements
System Requirements
| Specification | Minimum | Recommended |
|---|---|---|
| OS | Ubuntu 22.04, Windows 10 | Ubuntu 22.04 (Wayland) |
| CPU | Ryzen 2700K or Intel Core i7-8700K | Ryzen 5800X or Intel Core i7-11700K |
| Memory | 16 GB RAM | 32 GB RAM |
| GPU | NVIDIA RTX 3060 or better | NVIDIA RTX 4090 or better |
| Network | Broadband (used to download models) | Broadband (used to download models) |
| Storage | 22 GB (with models), 6 GB (without models) | 100 GB or higher |
Models
These are the sizes of the various models that power AI Runner.
| Model | Size |
|---|---|
| Controlnet (SD 1.5) | 10.6 GB |
| Controlnet (SDXL) | 320.2 MB |
| Safety Checker + Feature Extractor | 3.2 GB |
| SD 1.5 | 1.6 MB |
| SDXL 1.0 | 6.45 MB |
| LLM | 5.8 GB |
| e5 large (embedding model) | 1.3 GB |
| Whisper Tiny | 155.4 MB |
| Speech T5 (Voice) | 654.4 MB |
| OpenVoice (Voice) | 4.0 GB |
AI Models
By default, AI Runner installs essential TTS/STT and minimal LLM components.
You must supply additional Stable Diffusion models (e.g., from Hugging Face or Civitai).
Organize them under your local AI Runner data directory:
~/.local/share/airunner
├── art
│ └── models
│ ├── SD 1.5
│ │ ├── lora
│ │ └── embeddings
│ ├── Flux
│ ├── SDXL 1.0
│ │ ├── lora
│ │ └── embeddings
│ └── SDXL Turbo
│ ├── lora
│ └── embeddings
Unit Tests
To run all tests:
python -m unittest discover -s src/airunner/tests
Or a single test:
python -m unittest src/airunner/tests/test_prompt_weight_convert.py
Database
AI Runner supports a simple database system. See the Wiki for how to:
- Switch engines (SQLite, etc.)
- Make schema changes
- Run migrations
Advanced Features
- Memory Optimization: TF32 Mode, VAE/Attention Slicing, Torch 2.0, sequential CPU offload, ToMe token merging.
- Experimental Integrations: Weather-based chatbot prompts, advanced command-line arguments (
--perform-llm-analysis,--disable-setup-wizard, etc.). - Safety & Guardrails: Optional NSFW content detection and adjustable guardrails for LLMs.
Contributing
We welcome pull requests for new features, bug fixes, or documentation improvements. You can also build and share extensions to expand AI Runner’s functionality. For details, see the Extensions Wiki.
Take a look at the Contributing document and the Development wiki page for detailed instructions.
Thank You!
Thanks for checking out AI Runner.
Get started with local AI inference in minutes—no more endless environment setup.
Questions or ideas? Join our Discord or open a GitHub Issue.
Happy building!