You can run some of the biggest open AI models on an ordinary computer with colibri, a free open-source program that needs as little as 8 GB of RAM. The trick is that it keeps only the part of the model in use right now in memory and reads the rest from your disk. Setup is one command, but be realistic: the giant models run slowly, often a word every few seconds.
How can a huge model fit on a small computer?
Most of today’s biggest open models are “mixture-of-experts” models. Think of a hospital with hundreds of specialists: each patient only sees a few of them. These models work the same way, using only a small slice of themselves for each token they write. colibri keeps the always-needed core in RAM and pulls specialists from the disk only when called, caching the ones you use most. The project’s README notes that GLM-5.2 has 744 billion parameters but uses only about 40 billion per token.
What do I need before I start?
- At least 8 GB of RAM, though 16 GB or more is much better
- At least 22 GB of free disk space for the smallest model, and hundreds of GB for the giants
- A fast NVMe SSD if you want the big models, since disk speed sets their speed
- A graphics card is optional, colibri runs on the CPU and uses NVIDIA (CUDA) or other GPUs (Vulkan) when present
How do I install colibri step by step?
On Windows: open the colibri GitHub page, click Code, then Download ZIP, unzip it and double-click START-HERE.bat. It offers to install Python if it is missing.
On Mac: install Homebrew, then in Terminal run xcode-select --install, brew install libomp git python, git clone https://github.com/JustVugg/colibri, cd colibri and ./start-here.sh.
On Linux (Ubuntu or Debian): run sudo apt install git python3 build-essential, then clone the repository and run ./start-here.sh the same way.
The setup checks your RAM, disk and graphics card, recommends a model that fits, downloads it with resume support, and opens a chat dashboard in your browser at 127.0.0.1:8000. Next time, run the same start file and it opens straight away.
Which model should I pick?
Start small. On a normal laptop, the recommended pick is usually Qwen3.6-35B, a 23 GB download that ran at about 6 words per second on a desktop CPU in the project’s tests, and around 30 with an NVIDIA card. That feels like a usable chatbot.
The headline giants are a different experience. The README’s own measurements show DeepSeek V4 Flash at about 1 token per second on a 32 GB machine, Mistral Large 4 at about 1.3 with a good graphics card, and the 2.8-trillion-parameter Kimi K3 at about 9 seconds per token, with a 1.56 TB download. They work, but they are for patient experiments, not quick chats.
Can other apps use my local model?
Yes. colibri offers OpenAI-style and Anthropic-style web addresses on your machine, so many coding assistants and chat apps can point at it instead of a cloud service. It also has an MCP server so an AI assistant can install and manage it for you, asking before it downloads anything.
Why run AI locally at all?
Privacy and cost. Nothing you type leaves your computer, there is no monthly bill, and it keeps working offline. For a gentler first step with smaller models, see our guide to running an open-source model locally. As with any viral project, download only from the official repository.




