Pre-compiled wheels for llama-cpp-python on Windows. No Visual Studio, no CUDA Toolkit setup. pip install and run.
### New in this update
- **sm_120 (consumer/workstation Blackwell) support.** A single wheel now covers both sm_100 (datacenter) and sm_120 (RTX 5090 / 5080 / 5070 / 5060 / 5050, RTX PRO 6000 / 5000 / 4500 / 4000 / 2000 Blackwell). - **llama-cpp-python 0.3.20** across all four architectures (Blackwell, Ada, Ampere, Turing). Brings Gemma 4 support via the updated llama.cpp core. - **One wheel covers Python 3.10 through 3.13.** The 0.3.20 builds use py3-none tagging, no more per-interpreter builds. - **Fixed three mislabeled 0.3.16 sm_86 wheels** that linked against the wrong CUDA cuBLAS. Properly-built replacement is available.
I got tired of fighting with Visual Studio and CUDA Toolkit every time I wanted to use llama-cpp-python on Windows, so I've been building pre-compiled wheels for the community.
## What's Available: âś… RTX 50/40/30/20 Series support (Blackwell, Ada, Ampere, Turing) âś… CUDA 11.8, 12.1, 13.0 (Blackwell is CUDA 13 only) âś… Python 3.10-3.13 âś… Just 'pip install' and run - no build tools needed
## Why this matters: Windows users face a painful setup process with llama-cpp-python. These wheels eliminate: - Visual Studio installation - CUDA Toolkit setup - Compilation errors - Hours of troubleshooting
Why I think local, open-source models will eventually win.
The most useful AI applications are moving toward multi-turn agentic behavior: systems that take hundreds or even thousands of iterative steps to complete a task, e.g. Claude Code, computer-control agents that click, type, and test repeatedly.
In these cases, the power of the model is not how smart it is per token, but in how quickly it can interact with its environment and tools across many steps. In that regime, model quality becomes secondary to latency.
An open-source model that can call tools quickly, check that the right thing was clicked, or verify that a code change actually passes tests can easily outperform a slightly “smarter” closed model that has to make remote API calls for every move.
Eventually, the balance tips: it becomes impractical for an agent to rely on remote inference for every micro-action. Just as no one would tolerate a keyboard that required a network request per keystroke, users won’t accept agent workflows bottlenecked by latency. All devices will ship with local, open-source models that are “good enough” and the expectation will shift toward everything running locally. It’ll happen sooner than most people think.
