Underdog, a local AI assistant backed by a16z, has released Saluki 27B, a model built on Alibaba’s Qwen3.8-27B. Using roughly 2-bit quantization, the company said it cut the model size from about 54GB in full precision to 7.89GB. According to Underdog, the model can run on a laptop with 16GB of memory and can be deployed with existing tools such as llama.cpp. The weights have been released under the Apache 2.0 license. Saluki builds on quantization work from ISTA-DASLab at the Institute of Science and Technology Austria, which had previously compressed Qwen3.8-27B to about 8.4GB. In Underdog’s disclosed tests, Saluki answered 88 out of 120 tool-calling questions selected from BFCL, compared with 76 for the ISTA quantized version and 84 for the 54GB original. On 50 real GitHub bug-fix tasks, Saluki solved 30 while the original solved 33. Underdog said Saluki retained 96% of the original model’s performance on average across nine benchmarks, though math competition and complex reasoning scores were lower at about 83% and 85% of the original, respectively.
Underdog, a local AI assistant backed by a16z, has released Saluki 27B, a model built on Alibaba’s Qwen3.8-27B. The company said it used roughly 2-bit quantization to reduce the model from about 54GB in its full-precision version to 7.89GB.
According to Underdog, Saluki 27B can run on a laptop with 16GB of memory and supports deployment through existing tools including llama.cpp. The model weights have been open-sourced under the Apache 2.0 license.
Saluki uses quantization work from ISTA-DASLab at the Institute of Science and Technology Austria. That lab had previously compressed Qwen3.8-27B to about 8.4GB. Underdog said it pushed the size lower from there and focused on improving tool-calling performance.
In results released by the company, Saluki answered 88 out of 120 tool-calling questions selected from BFCL. The ISTA quantized version got 76 correct, while the 54GB original got 84. In 50 real GitHub issue-fixing tasks, Saluki solved 30 and the original solved 33.
Underdog said Saluki retained 96% of the original model’s performance on average across nine benchmarks, although the gap varied by task. On math competition and complex reasoning tests, the model reached about 83% and 85% of the original’s scores, respectively.
This article was originally published by Bit.Fan. For more cryptocurrency news and market insights, visit www.bit.fan. Disclaimer:
The market information, project data, and third-party content displayed on this platform are for industry information sharing only and do not constitute any form of investment advice or return commitment.
Cryptocurrency trading carries high risks. Users should fully assess their risk tolerance and make independent decisions. All profits, losses, and legal responsibilities are borne by the users themselves.
