A team from the Hong Kong University of Science and Technology and Microsoft, along with other collaborators, has released HarnessSQL, an open-source framework for training long-horizon interactive SQL agents on real databases. The work targets a problem that shows up quickly in enterprise analytics: large language models rarely produce a correct SQL query in one shot when the database contains hundreds of tables, thousands of fields, and millions of rows.
The paper says most traditional Text-to-SQL systems reduce the task to a single-pass generation problem. A model receives a question and a static schema, then outputs a final SQL statement. In practice, experienced analysts do not work that way. They inspect available tables, read DDL definitions, run probe queries, look at runtime errors, and revise their logic before submitting code. According to the authors, this is why static fine-tuned models struggle on long-horizon interactive benchmarks such as Spider2.0, BIRD-Interact, and LiveSQLBench: they were trained to write the endpoint answer, but not to explore, track state, recover from errors, or operate inside an execution harness.
Framing interactive Text-to-SQL as a POMDP
The paper formalizes interactive Text-to-SQL as a partially observable Markov decision process mediated by an execution harness. A task instance is defined as a four-tuple x = (q, D, h, v), representing the natural-language instruction q, the underlying database D, the execution harness h, and a hidden execution oracle v.
At each interaction step t ∈ [1, T], the agent observes o_t∈O and uses policy π_θ to generate reasoning and tool-use action a_t∈A. The final SQL output y is submitted at the terminal step. Success is measured by a joint reward function. In the paper, R_result(y, v)∈{0, 1} checks whether the candidate output and the reference answer are extensionally equivalent on the real database D, while R_protocol(τ, h)∈{0, 1} verifies whether the trajectory follows the harness contract, including read-only access, syntactic validity, no infinite loops, and explicit submission within T steps.
The authors argue that most post-training pipelines still rely on static supervised losses centered on the final SQL only. That creates three failures: intermediate exploration states disappear from training, the model never sees execution errors or empty result sets before deployment, and it never learns the action boundaries or termination rules of a database CLI.
Three building blocks in HarnessSQL
HarnessSQL is built around a high-fidelity closed execution sandbox, a minimal SQL-specific harness contract, and a two-stage long-horizon post-training pipeline.
Seventy-nine database sandboxes
The team constructed 79 complex database sandboxes using 30 SQLite databases from Spider 2.0-Lite and 49 DuckDB databases from Spider 2.0-DBT, with the latter migrated losslessly to SQLite. Data generation follows what the paper calls a SQL-First blueprint synthesis method. The system first extracts schema information, sample values, and foreign-key paths from a live engine, samples executable SQL queries with controllable difficulty, and then generates natural-language questions and validation oracles in reverse.
These sandboxes are configured with read-only binding, a single-statement constraint, and a 60-second timeout. The paper says syntax checks, non-empty checks, and semantic mutation tests are used to maintain data quality.
A four-action SQL harness
Instead of using the broader Bash-style tool stacks common in general coding agents, the team adapted DeepSeek-Harness into a SQL-specific contract called DSH-SQL. It contains four actions:
- sql_list_tables, which enumerates all tables in the database and their row counts;
- sql_schema, which inspects a target table’s full DDL, field types, and real sample data;
- sql_exec, which runs a read-only probe query and returns either leading rows or a runtime error stack;
- sql_submit, which submits the final SQL output and exits the session.
The middleware also adds three guardrails: deterministic row truncation to keep large outputs from overwhelming context, a loop-call warning mechanism to correct repeated ineffective queries, and pressure-triggered context compression to preserve key facts during long explorations.
Two-stage post-training: SFT first, RL after
The first stage is full-trajectory long-horizon supervised fine-tuning. A teacher model explores inside the sandbox, and trajectories that pass oracle verification are selected for training. The paper reports an average trajectory length of 11.8K tokens, 16 interaction turns, and 14 real executions.
A key detail is that all environment observation tokens are masked out during loss computation. Gradients are applied only to the model’s own generated reasoning, tool calls, and final submission. The authors say this forces the model to infer the next action from feedback rather than memorize a static answer pattern.
The second stage is execution-reward reinforcement learning. HarnessSQL uses Dynamic Decoupled Clipping Policy Optimization, or DAPO, to continue alignment inside the sandbox. Rewards come entirely from hidden-oracle execution comparison: a trajectory gets 1 if the result is equivalent and protocol-compliant, and 0 if it errors out, times out, or returns a mismatched result. The paper does not use handcrafted intermediate reward shaping.
Benchmark results
The team evaluated the system on three real-database benchmarks: Spider 2.0-SQLite for in-domain long-horizon interaction, BIRD-Interact Mini for out-of-domain dynamic interaction with a user simulator, and LiveSQLBench Base-Lite SQLite for cross-domain reasoning. Baselines include Qwen2.5-Coder-32B, OmniSQL 7B and 32B, Arctic-Text2SQL 7B and 32B, and the multi-agent system MARS-SQL 3×7B. All models were measured with execution accuracy.
On Spider 2.0-SQLite, HarnessSQL-8B reached 45.2% execution accuracy. That is nearly three times the 15.5% result of the same-base baseline and well above OmniSQL-32B at 14.8%, Arctic-Text2SQL-32B at 16.3%, Qwen2.5-Coder-32B at 16.3%, and MARS-SQL at 12.6%. Scaling to Qwen3-14B pushed the score to 54.8%.
The paper also reports strong out-of-domain generalization. On LiveSQLBench, the 8B model scored 24.8% and the 14B model scored 28.5%, while all 32B baselines fell in the 7.0% to 8.9% range. The authors interpret this as evidence that the model learned reusable exploration strategies rather than memorizing schemas.
Why SFT before RL matters
The paper compares direct reinforcement learning against a pipeline that starts with trajectory fine-tuning. For the 8B base model, direct RL raised execution accuracy only from 15.5% to 20.0%, a gain of 4.5 points. The authors say sparse binary rewards are hard to assign across long trajectories when the model lacks an interaction prior.
With verified full-trajectory SFT, the 8B and 14B models rose to 30.4% and 37.8%, gains of 14.9 and 15.6 points. Adding DAPO on top of SFT then lifted the 8B model by another 14.8 points to 45.2%, and the 14B model by another 17.0 points to 54.8%. The paper presents this as a practical recipe for long-horizon agents: SFT builds the interaction scaffold, RL improves adaptation in unseen environments.
Four ablation findings
Training and inference interfaces need to match
When inference used the DSH-SQL interactive environment but training relied on non-harness SFT data, Spider2.0 accuracy dropped to 9.6%, below the untuned base model’s 15.5%. Native Harness SFT reached 30.4%. The paper argues that static training can damage coordination with a dynamic environment, so training and deployment should remain structurally aligned.
Full trajectories beat turn-split supervision
Keeping the full trajectory context during SFT outperformed turn-split supervision across all three benchmarks. The authors say long-horizon tasks depend on causal continuity across many interaction rounds, and splitting samples weakens that continuity.
Longer context windows are critical
Real database exploration produces dense schema information and intermediate outputs, often pushing trajectories into the tens of thousands of tokens. Expanding training and inference context from 8K to 16K and then 32K raised Spider2.0 accuracy from 27.4% to 45.2%, according to the paper. Truncating early history removes the model’s ability to revisit prior evidence.
DAPO led under sparse rewards
Under the same budget and sandbox setup, DAPO scored 45.2%, ahead of GRPO at 40.7% and GSPO at 43.7%. The paper attributes the result to DAPO’s dynamic filtering of zero-variance trajectory groups, decoupled clipping ranges, and full-batch token normalization, which together reduce policy instability under sparse rewards.
What comes next
The authors outline two future directions. One is support for more database dialects beyond the current SQLite setup, including Snowflake, BigQuery, and PostgreSQL, along with stronger permission isolation and ecosystem adaptation. The other is adding multimodal and broader analytics tools, such as charts, reports, and interactive BI dashboards, instead of limiting the environment to command-line querying.
Paper and code are public
The paper argues that large models are moving from conversational assistants toward autonomous execution agents. In that setting, training on static “correct answers” is not enough when the task depends on tool use and trial-and-error feedback. The interaction environment itself has to be part of the training loop.
HarnessSQL’s code, sandbox environment, and synthetic trajectory assets have been released on GitHub. The paper is available at https://arxiv.org/abs/2610.12274, and the code repository is https://github.com/YangHaolin0526/HarnessSQL.
This article was originally credited to the WeChat public account "Xinzhiyuan," with authorship listed as Xinzhiyuan and editing by LRST.
