Open-weight AI models now trail the best closed models by only about four months on average, according to a Decrypt guide that walks readers through running models directly on a personal computer. The piece says a growing number of these models can run on hardware many users already own, using free tools such as Ollama and LM Studio.
In the guide, “open-weight” refers to models whose weights — the files containing what the model learned during training — are published for anyone to download and run. Decrypt says Ollama and LM Studio handle most of the setup work for end users. In Ollama’s library, Gemma 4 downloads range from about 4.6GB to 20GB.
That setup can remove the need for a roughly $20-per-month AI subscription and avoids sending conversations to someone else’s servers. Decrypt’s article focuses on how to download and run models on a local machine, what hardware is required, and how to get started with free software.
The guide says the appeal is straightforward: lower recurring cost, more privacy, and access to models that users can customize themselves.
Why some users want local AI
A cloud chatbot sends each prompt to a company server. Decrypt notes in the guide that, in its earlier review of privacy-focused AI tools, it reported that a court ordered OpenAI to preserve ChatGPT logs, including deleted ones. A local model keeps the conversation on the user’s computer, where nobody can access it without permission.
LM Studio’s documentation says the app “can operate entirely offline, just make sure to get some model files first,” and adds that documents attached to a chat can also be used fully offline.
On pricing, the article says paid chatbot subscriptions run about $20 a month, while the software discussed in the guide is free. The spending shifts to hardware, which some users may already have.
What users are allowed to do with downloaded weights depends on the license. Decrypt says DeepSeek’s R1 series is released under the MIT License, and its model card says the models “support commercial use, allow for any modifications and derivative works.” Google’s Gemma 4 and OpenAI’s gpt-oss are listed as shipping under Apache 2.0.
With licenses such as Apache or MIT, users can use and modify a model and build versions tailored to specific needs. Decrypt gives the example of someone who wants a model tuned for medicine and does not care much about coding performance. That kind of personalization, the article says, is available through customized models.
What consumer hardware can realistically run
Model size is usually described in parameters, the internal values adjusted during training. As parameter counts rise, download sizes usually rise with them. To run locally, a model needs to fit in a graphics card’s VRAM or, on a Mac, in unified memory shared by the processor and graphics hardware.
Decrypt gives a rough rule of thumb: at least 8GB of VRAM, with smaller compressed models able to fit. For heavier use, the guide says users should look at gaming GPUs with 24GB of VRAM or more, either by stacking GPUs or by buying dedicated AI racks or unified-memory systems such as the Mac Mini or Nvidia DGX Spark.
The article also explains quantization, which reduces model size by storing each weight with fewer bits. Citing Hugging Face documentation, Decrypt says this means representing weights and activations with lower-precision data types such as 8-bit integer, or int8, instead of the usual 32-bit floating point, or float32. The tradeoff is lower memory use at the cost of some accuracy.
LM Studio’s system requirements recommend 16GB or more of RAM on Apple Silicon Macs running macOS 14 or newer, though 8GB Macs can still handle smaller models. On Windows, the app recommends at least 16GB of RAM, 4GB of dedicated VRAM, and AVX2 processor support. Intel-based Macs are not supported.
Decrypt also lists several model footprints. Ollama’s library puts Gemma 4 26B at 16GB to 19GB and the 31B version at 19GB to 20GB. OpenAI says gpt-oss-20b needs only 16GB of memory, while gpt-oss-120b can run within 80GB.
How to set up LM Studio and Ollama
The guide says LM Studio will likely be the default choice for many users because it offers a polished graphical interface. Ollama is positioned more toward developers and users comfortable working in a terminal.
For LM Studio, Decrypt says users can download the installer for their operating system from lmstudio.ai, search for a model inside the app, download the weights, and open a chat. The app searches Hugging Face, which the article describes as the public hub where open models are published.
For Ollama, macOS and Linux users can install with curl -fsSL https://ollama.com/install.sh | sh. Windows users can run irm https://ollama.com/install.ps1 | iex in PowerShell. Decrypt says both commands are listed on Ollama’s download page.
Once installed, typing ollama pull gemma4:e2b downloads a model, and ollama run gemma4:e2b opens a chat. The command /bye exits. According to Ollama’s quickstart, the program also opens a local API at http://localhost:11434, allowing other apps on the same machine to use the model.
The guide warns that model names ending in “:cloud,” such as gemma4:cloud, are cloud models that run on Ollama’s infrastructure and do not require a download. Users focused on privacy should skip those. Decrypt adds that they can still serve as a second option for people who need more powerful models, because calls and prompts handled over the API are not stored on providers’ servers and are not used to train models.
Google’s Multi-Token Prediction drafters for Gemma 4 are also available on Ollama under Apache 2.0. Decrypt says they can speed up generation by up to 3x through speculative decoding, where a smaller model proposes several tokens in advance and the larger model checks them together.
Google said that “if the target model agrees with the draft, it accepts the entire sequence in a single forward pass,” and that the target model adds one extra token of its own in that same pass.
Local models can also gain web search and file access through MCP servers, short for Model Context Protocol. Decrypt says LM Studio added MCP support in version 0.3.17.
Where local models still fall short
The four-month average gap applies to the most capable open models, not to every model on the market. The largest systems remain far beyond consumer hardware. Decrypt points to Alibaba’s Qwen3.8-Max, which has 2.4 trillion total parameters and 95 billion active at any time. That stands in contrast to the 80GB memory envelope OpenAI cited for gpt-oss-120b.
If a model spills out of VRAM into system RAM, response times can slow sharply. Larger context windows also require more memory. Decrypt says Gemma 4’s smaller models offer a 128K-token context window, while the larger ones offer 256K.
Cloud services remove the need to plan around those hardware limits, and the frontier models behind them still lead on the hardest tests.
For users who want private AI workflows, Decrypt says its own review recommended “xPrivo with a self-hosted local model for day-to-day drafting that must not leave your machine,” and Kagi “when you need web retrieval and accept the API call tradeoff.”
