HKUST and Microsoft-backed team open-sources HarnessSQL, with 8B model beating 32B baselines on interactive SQL benchmarks
A team from the Hong Kong University of Science and Technology and Microsoft, among others, has introduced HarnessSQL, an open-source training framework built for long-horizon interactive SQL agents operating on real databases. The paper argues that conventional Text-to-SQL systems are trained to jump straight to a final answer from a static schema, while real analysts inspect tables, read DDLs, run probe queries, and revise logic after seeing errors. That gap between training and deployment becomes visible on interactive benchmarks such as Spider 2.0, BIRD-Interact, and LiveSQLBench. HarnessSQL addresses the problem with three pieces: a closed execution sandbox, a minimal SQL-specific harness with four actions, and a two-stage post-training pipeline that combines full-trajectory supervised fine-tuning with execution-reward reinforcement learning. On Spider 2.0-SQLite, the 8B version reached 45.2% execution accuracy, nearly triple the 15.5% base model result and ahead of several 32B baselines, including OmniSQL at 14.8% and Arctic-Text2SQL at 16.3%. The 14B version reached 54.8%. The paper also reports stronger out-of-domain generalization, gains from full-trajectory supervision over turn-split training, and a clear lift from expanding context length from 8K to 32K. Code and paper are publicly available on GitHub and arXiv.
