Ethereum co-founder Vitalik Buterin has introduced a plan to deploy localized private large language models (LLMs), with a strong focus on data privacy and security. The initiative aims to minimize risks of data leaks and unauthorized access by avoiding remote models and third-party services.
Key Strategies: Localization and Isolation
Buterin's approach includes local inference, on-device file storage, and sandbox isolation. This allows users full control over sensitive data interacting with AI, eliminating reliance on cloud servers. For industries like finance and healthcare, local private LLMs can significantly reduce compliance burdens.
Hardware Tests and Performance
To validate the concept, Buterin tested several hardware setups: a laptop with an NVIDIA RTX 5090, an AMD Ryzen AI Max Pro device featuring 128GB unified memory, and the NVIDIA DGX Spark. On the Qwen3.5 35B model, the 5090 achieved 90 tokens/second, the AMD device reached 51 tokens/s, and the DGX Spark hit 60 tokens/s. The results show high-end laptops offer the best balance for local AI environments.
Tools and Preferences
Buterin recommends using tools like llama-server, llama-swap, and NixOS to set up local AI environments. He personally prefers powerful laptops, citing a good trade-off between power consumption, portability, and performance. This setup also enables blockchain developers to run AI agents offline without relying on external networks.