Most RL-based prompt rewriters assign image-derived credit at the subword level, but visual requirements (objects, counts, attributes, relations) naturally form coherent semantic spans. That mismatch dilutes learning signals for multi-token concepts. The core insight behind this work is that grouping token-level policy shifts by semantic span and sharing update weights within each span yields more targeted credit assignment and better prompt rewrites for text-to-image (T2I) systems.
Key Findings
- Semantic-span pooling: The rewriter jointly predicts an enhanced prompt and its semantic partition; SCOPE pools policy log-probability shifts within each span to derive a single coefficient that scales token losses inside that span, while keeping one advantage per completion. This concentrates updates on coherent concepts rather than individual subwords.
- GRPO-based objective: Builds on Group Relative Policy Optimization (GRPO) to produce a semantic-aligned update rule that requires no separate span rewards and no extra image renders.
- Empirical gains: Across multiple generators (Qwen-Image, SD3, FLUX.1-dev) SCOPE yields average relative improvements over vanilla GRPO of about 9.48% on GenEval and 7.62% on T2I-CompBench. Ablations confirm both boundary prediction and within-span pooling contribute to gains.
- Efficiency: Achieves improvements without additional rendering cost or per-span reward engineering, making it practical for rewriter-in-the-loop workflows.
Who it's for and tradeoffs
Great fit if you train or fine-tune prompt rewriters for frozen T2I generators and want stronger, semantically targeted credit assignment without increasing rendering budget. It is especially relevant when prompts contain multi-token entities, attribute bindings, or relational constraints that benefit from group-wise updates. Look elsewhere if you need a method that also adapts the image generator itself (SCOPE targets the rewriter only), or if your pipeline cannot generate or use semantic-span partitions for prompts.
How it works (brief)
The pipeline lets a language-model-based rewriter output both a rewritten prompt and a segmentation of that prompt into semantic spans. During RL updates, token-level policy log-probability shifts are pooled per span to compute a shared coefficient that weights the token losses for that span. The approach preserves per-completion advantages but replaces per-token weighting with span-level grouping, yielding more coherent update signals for multi-token semantics.
This work highlights semantic structure as a practical inductive bias for prompt engineering with RL and demonstrates measurable gains with minimal operational overhead.