Keep the model fixed. Redesign the surface it thinks through.
A coding agent does not act on a repository directly. Something decides which commands it may issue, how a file is shown, what an invalid edit does, and which old observations survive into the next turn. The paper names that whole layer the agent-computer interface, or ACI.
SWE-agent treats this interface as a design variable. Its bet is modest but consequential: give a language model compact actions, concise feedback, and guardrails shaped around its weaknesses, and the same model can navigate and edit a repository more reliably.
For an LM, the interface is part of the reasoning system.
The missing layer
A model, a repository, and the machinery between them
Step 1 · begin with the task
A real fix is a conversation with a changing repository.
Let us begin where Figure 3 begins: inside a 3,093-line file, with four exact lines that need a more careful condition.
In the paper's worked trace, the agent has opened
solvers/diophantine.py. It notices that the branch for
total_degree > 3 should also check that all powers
share the same exponent before assigning
general_sum_even. The relevant code sits at lines
404–407.
This is already unlike a short code-generation puzzle. The model must see a small piece of a large file, name an exact edit, observe what changed, and continue from the new state. In the diagram, notice that the arrow returns from the repository to the model; one-shot patch generation has no such return path.
Figure 3 supplies the four-line edit we will keep returning to. The loop around it is the object SWE-agent redesigns.
We assume only that an LM can generate commands, a shell can run them, and a submitted patch must pass tests. ReAct, HCI, and SWE-bench will be unpacked as they appear.
Step 2 · why the shell is not neutral
Freedom at the command line can become bookkeeping for the model.
Our target is only four lines, but they live in a 3,093-line file. How the model reaches and changes those lines is now the problem.
A human can mix cat, grep,
sed, line ranges, and editor state almost without
noticing the coordination cost. The paper finds that current LMs
often cannot. Printing a whole file floods context; a silent
sed command leaves the model unsure whether the edit
landed; multi-line edits demand fragile quoting and arithmetic.
Point at the left card below: the model must remember the file path, recalculate ranges, write a shell expression, and then ask for the file again. The right card bundles the intended operation and its immediate feedback into one action.
Human-oriented shell
$ cat solvers/diophantine.py [3,093 lines enter context] $ sed -i '404,407c\...' file.py [no output] $ sed -n '400,412p' file.py [verify in another action]
LM-oriented action
edit 404:407
elif (total_degree > 3 and
len(set(...)) == 1):
if all(coeff[k] == 1):
diop_type = 'general_sum_even'
end_of_edit
→ updated file window
This is not an argument that Bash is bad. It is an interface built for a different user. The paper's claim is narrower: a model has different strengths, costs, and failure modes than a human terminal user.
Step 3 · name the missing layer
An ACI designs both what the agent can do and what it sees next.
The two edit cards differ in more than command syntax. One also controls the return message, file state, and history carried forward.
The paper defines the agent-computer interface as the layer through which an LM uses a computer. It specifies actions and their documentation, formats environment feedback, tracks prior interactions, and combines that history with task instructions for the next model call.
SWE-agent's loop uses the ReAct pattern: each turn contains a thought and one action; the environment executes the action; an observation—the computer's returned state description—feeds the next turn. Look at the middle box: commands and observations travel in opposite directions, but both belong to the ACI.
First zoom: the ACI is not just the amber command box. It also owns the feedback and history that close the loop.
Find the file and line
Search broadly, open a focused window, then move to an exact location.
search_dir → open → goto
Construct a candidate state
Replace a line interval, check it, and return the changed region immediately.
edit 404:407
Learn from execution
Run reproduction code or tests, then decide whether to revise or submit.
python / pytest → submit
Simplification boundary: this three-phase view folds prompts, the Docker environment, configuration, and logging into one picture. Appendix A and C open those implementation layers.
One checked turn
Observe → replace → validate → commit or roll back → report
Step 4 · observation budget
A useful file view shows enough context without making every line compete.
We now know the route—localize, edit, test—but our running file has 3,093 lines. The first concrete interface choice is how much of it to show.
SWE-agent's file viewer opens at most 100 lines at a time. The
observation includes the full path, total line count, visible line
numbers, and how many lines sit above and below. The agent can
scroll or jump with goto without rebuilding a shell
command.
Drag the three-stop control. The code window changes as a visual model of the observation budget; the three resolve rates are the paper's measured SWE-bench Lite ablation. Notice the middle stop: among the three tested settings, 100 lines performed best.
Use the arrow keys for one exact stop at a time
Read the three stops as three separate experiments. They do not define a smooth curve, and 18.0% is an aggregate Lite result—not the success probability of this file.
Step 5 · exact replacement
The edit command turns a vague intention into one checkable splice.
The 100-line view pins down 404–407 and prints their line numbers. That makes the next action short enough to inspect by eye.
SWE-agent's edit n:m replaces the inclusive interval
from line n through line m in the open
file. Here, 407 − 404 + 1 = 4 old lines leave the
file, and four replacement lines enter.
We can therefore compute the candidate length before discussing
any abstraction: 3,093 − 4 + 4 = 3,093. In the right
window below, point to line 405—the new exponent check now shares
the outer condition with total_degree > 3.
The paper's action · Figure 3
edit 404:407
elif (total_degree > 3 and
len(set(k.exp[k.is_Pow])) == 1):
if all(coeff[k] == 1):
diop_type = 'general_sum_even'
end_of_edit
The arithmetic is deliberately small: an inclusive four-line interval becomes four lines, so later line numbers do not shift. The candidate exists now—but it has not earned a commit yet.
Step 6 · commit or roll back
A guardrail lets the model fail against a candidate state, not the repository.
The splice constructed a candidate file. “Generated” is not the same as “accepted”; the ACI still has to decide which state survives.
For Python edits, SWE-agent runs selected flake8
checks after applying the candidate. If no selected error appears,
it keeps the edit and shows the updated region. If an error
appears, it restores the original file and returns three pieces:
the error, the would-be edited snippet, and the original snippet.
Our Diophantine edit succeeds, so the paper changes examples when
it needs to show a rollback. Figure 11 removes an
orientation_strategy argument while references remain;
two F821 errors fire, and the candidate is discarded.
Use the three-stop editor control to compare the exact Lite
configurations reported in Table 3.
Each stop is a discrete interface configuration
Selected linter finds 2 × F821
The proposed signature omits orientation_strategy
while the function body still uses it.
F821 undefined name F821 undefined name
Original file restored
The invalid candidate never becomes the next repository state. The observation asks the agent to revise its edit.
S(t+1) = S(t) observation = error + proposed + original
With linting, the ACI rejects the malformed candidate and gives the next model call enough local evidence to try again.
The guardrail checks form, not meaning. Its selected errors include syntax, indentation, undefined names, and related codes; it cannot tell whether a syntactically valid patch solves the issue.
The paper also names a trade-off: rejecting intermediate undefined references can force a multi-step refactor into a particular edit order. A guardrail reduces common errors but also narrows the workflow.
Step 7 · write one full state update
The next turn is built from a validated repository and a deliberately shaped observation.
We can now distinguish candidate, commit, and rollback. Let us return to lines 404–407 and execute the successful branch from end to end.
This is the page's slowest step because two kinds of “memory” are easy to mix up. The LM's weights do not change. The repository state changes when a candidate passes, while the conversation history gains an observation describing that result.
Walk down the worksheet. The four-line action produces a
3,093-line candidate; selected lint returns no error; the candidate
becomes S(t+1); the updated file window becomes
o(t+1). That is one minimal ACI update you can now
reproduce without a black box.
Teaching notation, not an equation printed by the paper. It compresses the editor behavior described in §3 and Appendix A.1.
Second zoom: the simple bridge from Step 3 now contains the candidate, validator, two state branches, observation formatter, and history processor.
SWE-agent keeps the latest five observations in full and collapses each earlier observation to one line. “Last 5” does not mean the older action history vanishes; it means bulky old outputs stop competing with the current file state.
Evidence and limits
Component ablations first, headline scores second, failure modes last
Step 8 · test the interface claim
Small ACI choices corresponded to large behavior changes in the Lite ablation.
Our four-line turn makes the proposed mechanism visible. The 300-task Lite study asks whether those choices matter beyond one trace.
The default SWE-agent configuration with GPT-4 Turbo resolved 18.0% of SWE-bench Lite. The authors then changed one interface component at a time. These are not decorative settings: each one changes what information enters a later model call or how easily an action can advance repository state.
Read the red bar in each card as the reference configuration.
The iterative search result is especially revealing: stepping
through matches one by one led agents to exhaustively call
next, scoring 12.0%—lower than having no custom search
at all, at 15.7%.
Editor
Does one compact action validate its candidate?
Search
How are matches packaged for the next turn?
File viewer
How much of the open file is returned?
Context
How much old observation text stays expanded?
The authors summarize the pattern as simple actions, compact higher-order operations, informative but concise feedback, and guardrails that shorten recovery from common mistakes.
Table 3 uses one 300-task split and the paper's 2024 model/configuration. It supports the interface claim within that setup; it does not prove that 100 lines or “last five” is universally optimal.
Step 9 · the complete system
With the full ACI, a fixed base model crossed from patch suggestion to iterative repair.
The ablations isolate interface pieces. The full SWE-bench run asks whether the assembled loop resolves real repository issues end to end.
On all 2,294 SWE-bench test issues, SWE-agent with GPT-4 Turbo resolved 12.47%, or 286 tasks. On the 300-task Lite split it resolved 18.0%, compared with 11.0% for the Shell-only agent using the same base model and 2.67% for the GPT-4 Turbo RAG baseline.
A resolve rate is the share of tasks whose generated patch passes all evaluation tests after being applied. It is stricter than producing plausible code. In the report below, point to the first two bars: the same GPT-4 Turbo moves from 11.0% with the Shell-only interface to 18.0% with SWE-agent on Lite.
SWE-bench Lite · % resolved · GPT-4 Turbo unless noted
Cost is part of the trade-off. Averaged over successfully resolved Lite instances, the paper reports an API cost of $0.13 for GPT-4 Turbo RAG, $1.46 for Shell-only, and $1.67 for SWE-agent. Each task had a $4 budget. These are historical experiment results, not a current leaderboard or present-day price.
Step 10 · keep the unsolved part visible
A better interface reduced friction; it did not make correctness automatic.
The 12.47% full score was a substantial 2024 jump, and it also leaves 87.53% unresolved. The failure analysis tells us where interface help ends.
Among 248 unresolved Lite trajectories, 52.0% were categorized as an incorrect or overly specific implementation. Another 23.4% failed to recover from repeated edits. Across all 2,294 GPT-4 Turbo trajectories, 51.7% had at least one edit rejected by linting. The system often found the right neighborhood and still wrote the wrong fix.
The strip separates those two large failure groups from everything else. It also clarifies the guardrail's boundary: lint can reject an undefined name, but it cannot judge whether the exponent condition in our running example is the right general solution.
The failure labels were produced by GPT-4o. On a hand-labeled sample of 15 trajectories, its labels agreed with the authors 87% of the time. The study also developed ACI components manually and focused only on programmatic tasks; the paper names both as limitations.
Try answering before opening each card. The goal is not to memorize 12.47%; it is to keep candidate state, validated state, observation, and model weights separate in your head.
What exactly does an ACI design?
Its action space and documentation, the format of environment feedback, guardrails around execution, and the history assembled for the next LM call—not merely a list of tools.
Why does edit 404:407 keep 3,093 lines?
The interval is inclusive, so it removes 407 − 404 + 1 = 4 lines. The replacement also has four lines: 3,093 − 4 + 4 = 3,093.
What changes when lint reports an error?
The candidate is discarded and repository state stays at S(t). The next observation contains the error, proposed snippet, and original snippet so the model can revise.
Did the model learn during the edit loop?
Not through weight updates. Repository state and conversational context changed across turns; the base LM weights remained fixed.
Why not conclude that 100 lines is always best?
The paper measured three viewer settings in one 2024 benchmark setup. The middle setting won there; three discrete points do not establish a universal law.
What does the 18.0% Lite result isolate?
The complete SWE-agent configuration. Component ablations and the 11.0% Shell-only comparison provide evidence that the interface—not only the fixed GPT-4 Turbo model—contributed.
One-minute redraw
Draw six boxes from memory: LM call → action parser → candidate file
→ selected lint → commit/rollback → formatted observation. Add one
arrow back to the model and label what changed.
The interface became a first-class research object.
SWE-agent's lasting proposal is not that one command set is final. It is that model capability should be studied together with the surface through which the model acts.
A new kind of end user
LMs have different memory costs, visual abilities, and error patterns than humans, so a human UI is not automatically a good agent interface.
Behavior becomes design evidence
With model weights fixed, command, feedback, and history ablations turn agent trajectories into evidence about interface compatibility.
The question travels
The paper asks whether search, shopping, data analysis, and knowledge work need their own ACIs—an HCI-like program for artificial users.
The number 12.47% will age. The instruction to evaluate a model and its interface as one acting system is the idea built to travel.