Local AI

17 hours agoNews Editor

How to Run AI Models Locally, What Hardware You Need, and Where Cloud Still Wins

Decrypt has published a detailed guide on running open-weight AI models directly on a personal computer, arguing that the gap between the best open models and top closed systems has narrowed to about four months on average. The article says users can now run a growing number of models on hardware they may already own, using free tools such as Ollama and LM Studio. The guide frames local deployment around three main tradeoffs: privacy, cost, and customization. Running a model on-device keeps conversations on the user’s machine instead of sending prompts to a company server, and it avoids the roughly $20-per-month subscription common among paid chatbot services. It also gives users access to models released under licenses such as MIT and Apache 2.0, which can allow commercial use, modification, and derivative work depending on the model. Decrypt also breaks down the practical limits. Smaller or quantized models may fit on systems with 8GB of VRAM, while heavier users may need 24GB or more, or machines built around large unified memory pools. The piece walks through installation steps for LM Studio and Ollama, notes support for local APIs and MCP servers, and points out that cloud services still hold the edge for the largest models and the hardest benchmarks.

10

Kimsuky

3 months agoNews Editor

Kimsuky Sets Up Local LLM Environment as AI Attack Capabilities Grow

On Aug. 10, cybersecurity firm Genians disclosed that Kimsuky, a hacking group tied to North Korea's Reconnaissance General Bureau, has expanded its use of generative AI. The group now runs a standalone local large language model (LLM) environment and a retrieval-augmented generation (RAG) system, moving beyond earlier work that used AI mainly to craft phishing lures. According to Yonhap, the tools involved include local LLM managers Ollama, GPT4All and Msty, along with the AI code editor Cursor. Genians said the shift points to an intent to automate intelligence extraction and attack workflows. The public disclosure ties the upgrade to Kimsuky's broader mission of gathering intelligence and running cyber operations. Building a local AI stack means the group can generate text and process technical information without leaning on commercial cloud services. The reported use of RAG also suggests an effort to pull data from a private knowledge base and turn it into usable output. No further details about the group's infrastructure were included in the announcement.

2350

Bit.Fan