Inco AI has released Splash, an open-source local inference engine for Mac, and said Qwen3.8-27B reached a peak of 144 token/s on an M5 Max MacBook Pro. LM Studio has already added support, with direct access available in version 0.4.25. The project builds on Inco’s earlier DFlash approach, which lets a smaller model guess multiple tokens in parallel before a larger model verifies them in batches, cutting the cost of token-by-token generation. Splash applies that optimization across the full local inference engine rather than as a narrower acceleration layer. For now, support is limited to Qwen3.8-27B and Qwen3.6-35B-A3B, with each model getting its own GPU kernel, memory setup, and DFlash 2 small model. Inco also shared benchmark figures from a 48GB M5 Pro used for side-by-side testing: Qwen3.8-27B hit 74 token/s in a single short-context stream, while aggregate throughput reached 170 token/s under four-way concurrency, or 3.9 times the runner-up. The company said that profile makes Splash better suited to Agent workloads, where total concurrent throughput can matter more than the speed of a single chat response.

Shanghai AI Lab and Shanghai Jiao Tong University have introduced NCP-ArchPreview, an 8.9 billion-parameter model that extends the usual next-token prediction setup by also forecasting internal concept signals for a short span of upcoming text. Those signals are then used to assist token generation, allowing the model to keep emitting output one token at a time while internally anticipating what comes next. The same mechanism is also applied to speculative decoding. In systems such as DFlash and DSpark, a smaller draft model predicts upcoming text before a larger model verifies it. The NCP team feeds internal concept signals to that smaller model as an added hint about how the continuation may unfold, then lets it predict 16 tokens in parallel. According to the reported tests, this improved how much of the draft the larger model accepted. Across four evaluations, the average number of tokens accepted per verification rose from 5.933 to 6.180, a 4.17% increase. On HumanEval, the gain reached 7.59%. Adding the signal increased the draft model by only about 40,000 parameters. At the same 8.9B scale, NCP also reduced training loss to a similar level using about 85% of the training compute required by a standard Transformer.