ByteDance Seed says a neglected variable in text-to-image training deserves to be treated as a scaling factor in its own right: how much image-grounded information a caption actually delivers to the model. The team’s central claim is straightforward. What scales with text conditioning is not the number of caption tokens by itself, but the amount of visual information the model can reliably use.
The work appears in the paper Scaling Properties of Text Conditioning in Visual Generation by Zilong Chen, Chaorui Deng, Kunchang Li, Hongyi Yuan, Haoqi Fan of ByteDance Seed. The team also released the paper, project page, code, model collection, and an online demo.
- Paper: https://arxiv.org/abs/2607.29679
- Project page: https://heheyas.github.io/context-scaling
- Code: https://github.com/heheyas/context-scaling
- Model: https://huggingface.co/collections/heheyas/context-scaling
- Demo: https://heheyas-context-scaling.hf.space/
- Hugging Face paper page: https://huggingface.co/papers/2607.29679
Longer prompts did not make models stronger
Text-to-image systems have advanced for years through familiar levers: larger models, more data, and more training compute. Seed argues that this framing misses a basic difference between image generation and language modeling. A language model can learn directly from text sequences through self-supervision. A text-to-image model learns from paired images and captions, mapping language to visual content.
An image may contain many objects, attributes, positions, actions, and relations. Only the parts described accurately and bound clearly in the caption become usable supervision through the text interface. That leads to the question behind the paper: instead of scaling only model size, data, and compute, can image generation improve by increasing the amount of image information packed into captions?
A natural first guess would be to write longer, more detailed training captions or user prompts. Seed reports that experiments point the other way. Across several open text-to-image systems, natural-language prompts saturated quickly as length increased, and final outputs in some cases fell below the results from the shortest prompts in each setup. Even when diffusion models were trained specifically on the same set of long-form captions, gains remained limited.
To isolate the effect, the team built a fixed-backbone image reconstruction experiment. For the same reference image, researchers started from one complete annotation and produced four natural-language captions with progressively more detail. They then used the same Qwen-Image model and the same random seed to reconstruct the image from each caption. The entities and relations stayed the same across versions; later captions mainly expanded the wording and prose.
The finding was that caption length rose sharply, while reconstruction quality barely improved after a point. The added prose often explained, paraphrased, or connected information already present, rather than introducing new visual variables the model could keep using in a stable way. In Seed’s framing, caption length is only a weak proxy. A text can be long without stating clearly which attribute belongs to which object, how two objects are related, where they are placed, or what front-to-back ordering the scene contains.
Two measures for the image information inside captions
If token count does not capture the strength of supervision, the next step is to measure the image-bound information in a caption directly. Seed adapts two complementary indicators for that purpose: Grounded Perplexity Gain, or GPG, and Effective Detailness, or ED.
GPG is a white-box metric that requires access to token probabilities. For the same caption, the researchers let a frozen vision-language model process the text once with the paired image and once without it, then measure the gain in log-likelihood for content tokens when the image is present. If a caption carries a large amount of information tightly grounded in the image, seeing the image should make those tokens much easier to predict.
ED is a black-box semantic metric that does not rely on token probabilities. It extracts attributes with entity context from the image and from the caption, computes caption-attribute precision and image-attribute recall, and then uses F0.5 as the final score so that unsupported details in the caption are penalized more heavily.
The two metrics look at the same issue from different angles. GPG tracks statistical dependence between image and text. ED measures whether the caption accurately covers verifiable visual content.
Caption information predicted converged diffusion loss
Seed then held the image set, model architecture, initialization, optimization configuration, and training budget constant, changing only the training captions. The full study covered 15 caption configurations: three natural-language versions of different lengths, six Structured Prompt variants with fields restored step by step, and six variants involving spatial-expression changes or field masking. Each configuration started from the same BAGEL continued-training checkpoint, and one diffusion model was trained independently for each setup.
Token count alone did not produce a unified relationship with training outcome. Once caption information was used on the horizontal axis instead, the configurations across different formats and detail levels lined up on stable curves.
- Converged diffusion loss was approximately linear with GPG, with Pearson r = -0.984.
- Converged diffusion loss followed a power-law trend with ED, with Pearson r = -0.971 in log-log space.
- GPG and ED also ranked the caption configurations similarly, with Spearman ρ = 0.96.
The team calls this the scaling properties of text conditioning. It does not present the result as a universal law for every model, but as an empirical calibration under a fixed architecture and training recipe.
That calibration has two practical uses in the paper. First, caption information becomes a controllable and measurable training variable rather than a vague label for data quality. With model, images, and compute held constant, that variable predicts the training loss a model can reach. Second, once the relationship is calibrated for a given recipe, GPG or ED can be used to compare candidate caption schemes before paying for a full diffusion training run. Seed says the two metrics still predicted converged loss reasonably well for six caption variants not used in the fit.
Structured Prompt organizes visual variables into named fields
The earlier experiments suggest that more prose is not enough by itself. The new information also has to be organized in a stable and explicit way. Seed’s answer is Structured Prompt, or SP, a JSON-based representation of the visual variables in an image.
The representation has three levels:
- Global level: scene intent, setting, mood, style, lighting, and photography information.
- Element level: each subject’s identity, attributes, actions, position, optional depth, and local photography details.
- Relation level: position, occlusion, interaction, and semantic relations across elements.
Seed argues that the important part is not the JSON surface form by itself. The real gain comes from assigning visual variables to stable named fields, which cuts ambiguity around attribute ownership, spatial relations, and object binding. In the fixed-backbone reconstruction test, quality kept improving as SP fields were restored step by step. In the full training sweep, broader field coverage also raised GPG and ED while pushing converged diffusion loss lower.
To generate full SP annotations at scale, the team built an image-to-SP pipeline. A general vision-language model handled global semantics and local content. Sapiens added evidence for human pose. DepthAnything V2 supplied relative depth. SAM 2.1 provided masks and occlusion cues. A vision-language model then merged those signals into a consistent full SP.
Diffusability and Promptability split the problem in two
Seed uses the term Diffusability for a caption representation’s ability to expose and organize image supervision for the diffusion model. That is the side SP improves: without changing the diffusion architecture, it lets the model learn more visual variables and learn them more explicitly from text conditioning.
Training time is one thing. In actual generation, the system does not get a reference image or oracle annotation, only a user request. That creates the second side of the problem. The system needs an LLM prompter that can expand a user request into a detailed, coherent SP without violating the original constraints. Seed calls this capability Promptability.
The paper expresses the relationship as:
Generation Quality = Diffusability × Promptability
The authors add that the multiplication sign is an organizing view, not a fitted mathematical formula. Diffusability describes what the diffuser can learn from a caption representation. Promptability describes whether an LLM can actually fill that representation with a high-quality instance at inference time.
Larger and better-trained prompters improved results with the same diffuser
To test the prompting side, the team fixed the SP schema and the Qwen-Image diffuser, then swapped only the zero-shot LLM prompter. As Qwen3.5 scaled from 0.8B to 397B, GenEval++ in thinking mode rose from 46.4% to 86.8%. The paper adds one caveat: the smallest model tended to repeat itself during reasoning and failed to output valid JSON, while chain-of-thought improved results at other scales.
That result, in the team’s reading, shows that stronger general-purpose LLM capability and stronger reasoning can pass through the caption interface and show up as better image generation.
Zero-shot LLMs still tended to produce SPs with limited information and simpler compositions. To raise Promptability further, Seed used a three-stage training setup:
- SFT to learn the SP content distribution expected by the diffusion model, not just the JSON format.
- Cold-start distillation from privileged reasoning traces with paired images, teaching how to infer SP from only the user request.
- RFT on the prompter’s own generated and rendered rollouts, with a verifier selecting high-confidence trajectories and an image-conditioned teacher providing dense token supervision through on-policy self-distillation.
Ablation results in the paper assign different roles to the three stages. SFT delivered the biggest single-stage structural gain. Cold-start strengthened the inference path from user request to SP. Verifier-gated OPSD produced the strongest results on the prompter’s own distribution.
Structured fields also made iterative repair easier
SP has another practical advantage. When the generated image is wrong, the system can revise the specific object, attribute, relation, or layout field involved instead of rewriting a full natural-language prompt from scratch.
Seed built a refine-render-judge loop around that idea. In each round, the prompter generates or revises the SP based on the user request and prior feedback. A fixed diffuser renders the image. An online judge then returns PASS or FAIL along with concrete issues tied to prompt following, structure, and visual quality. If the result fails, the next round edits only the relevant fields.
The paper reports that giving the loop more iterations continued to improve structure, alignment, and GSB performance, but useful reasoning length stayed short. For the trained prompter, even with up to eight rounds allowed, the average usage was only 2.31 rounds. Gains from raising Tmax from 4 to 8 were already small. Seed says that text-to-image generation benefits from iterative error correction, but under the current setup it does not need a long prompt-side reasoning trajectory. Once major specification errors are fixed, extra rounds saturate quickly.
With the same Qwen-Image setup, the structured interface outperformed free-form captions
The final system combines an SP-trained diffuser with the trained LLM prompter. Seed says it led the open-weight models used for comparison on nearly all reported metrics, and matched or exceeded the closed systems it compared against on most evaluations, with the clearest gains in compositional, reasoning, and world-knowledge tasks.
The paper also includes a matched control designed to rule out the simpler explanation that the structured system only benefited from more training. Using the same Qwen-Image architecture, the same training images, the same training stages, and the same budget, the researchers trained another system that always used free-form natural-language captions.
That Matched NL setup did improve somewhat with the extra training, but not nearly enough to reproduce the SP system’s results. Seed’s conclusion is that the gains cannot be explained simply by a larger backbone or more training. They are closely tied to the structured caption interface used between the prompter and the diffuser.
Scaling the condition itself enters the picture
The premise of the work is simple: in text-to-image models, captions are not incidental metadata. They are the main interface through which image content enters text-conditioned learning.
When a caption only gets longer, the added tokens may be little more than restatement and elaboration. When image information is extracted accurately, bound clearly, and organized consistently, the same generation model can learn more from it. In the paper’s framework, GPG and ED make that information measurable, Structured Prompt improves Diffusability, and LLM scaling, post-training, and short-horizon agentic refinement improve Promptability.
The paper’s closing takeaway is that the next step in text-to-image scaling should not focus only on how large the rendering model is. It should also ask how much image information the text condition actually carries that the model can learn from and use.
This article is based on a post originally published by the WeChat account Jiqizhixin and distributed by MarsBit.