An open-source project called kimi-k3-in-c aims to run Kimi K3, a model with 2.78 trillion parameters, on devices with just 8GB of memory. The 176KB codebase is written in pure C99 and performs inference on the CPU only, dropping GPU, CUDA, PyTorch, and BLAS entirely. The approach exploits Kimi K3's MoE architecture: only 16 of the 896 experts per layer are activated, so the developer avoids loading the full ~1.56TB of weights and instead streams most expert weights from NVMe storage on demand. Dense trunk layers are streamed layer by layer as well. Trade-offs remain: generating one token takes about 32.7 seconds, and close to 1.7TB of fast storage is needed. The developer describes the project as an experimental exploration of LLM inference infrastructure rather than a production-ready solution, but the combination of hard-drive streaming and sparse MoE activation points toward new ways to run ultra-large models at low cost.
On August 8, a developer open-sourced kimi-k3-in-c, a project designed to run Kimi K3 — a 2.78-trillion-parameter model — on devices with just 8GB of memory. The whole project is only 176KB, written in pure C99, and it performs inference on the CPU alone. No GPU, CUDA, PyTorch, or BLAS is involved.
The approach leans on the mixture-of-experts (MoE) architecture behind Kimi K3. The model has 2.78T total parameters, but only 16 of the 896 experts in each layer are activated during inference — most of the weights are simply not used at any given step. That's why the developer chose not to load the complete ~1.56TB of model weights into memory. Instead, the majority of expert weights live on NVMe storage and are pulled in on demand as inference proceeds; some dense trunk layers are streamed layer by layer in a similar fashion.
The performance cost is explicit. In 8GB memory mode, generating a single token takes roughly 32.7 seconds, and the setup needs close to 1.7TB of high-speed storage to work at all.
The developer calls the project an experimental exploration of how large-model inference infrastructure might be optimized, not a solution with real production value. Still, the combination of hard-drive streaming and sparse MoE activation offers a new direction for running ultra-large models at low cost in the future.
This article was originally published by Bit.Fan. For more cryptocurrency news and market insights, visit www.bit.fan. Disclaimer:
The market information, project data, and third-party content displayed on this platform are for industry information sharing only and do not constitute any form of investment advice or return commitment.
Cryptocurrency trading carries high risks. Users should fully assess their risk tolerance and make independent decisions. All profits, losses, and legal responsibilities are borne by the users themselves.
