An agent can reason correctly about costs and still explore poorly when it lacks a reliable estimate of what it knows. The useful distinction here is between estimating uncertainty and choosing an action: providing the former explicitly helps the same model make better decisions about the latter.
Key Findings
- On the controlled Pandora’s Box task, Qwen3-8B with explicit priors and thinking matched the optimal policy in 94% of cases, versus 23% for ordinary prompting. This isolates a gap between reasoning ability and access to usable uncertainty information.
- On PopQA with optional retrieval, calibrated confidence increased cost-discounted reward from 0.283 to 0.293. The objective rewards selective retrieval; it does not simply maximize answer accuracy at any cost.
- On the constructed FileReading task, adding estimated file-format priors to RL raised reward from 0.259 to 0.268, about 3.5%. Agents adjusted testing to relative execution costs rather than defaulting to testing first.
How Calibration Enters the Loop
CTA-Prompted supplies priors in the prompt; CTA-RL also trains with that information using GRPO. QA confidence is calibrated with isotonic regression, while a small filename-to-format predictor supplies priors for FileReading. This makes uncertainty estimation a separate component whose quality can be evaluated.
Who Should Read It
Great fit if you design retrieval or coding agents and need a framework for deciding whether another tool call is worth its cost. Look elsewhere if you need evidence from unrestricted production workflows: the study uses one main model, controlled tasks, explicit cost functions, and task-specific prior estimators. Better reward here supports the decision mechanism within those settings; deployment requires checking estimator quality and realistic costs.