RL Is Becoming Local: What Composer 2.5 Teaches Us About Long-Horizon Agents
Cursor's Composer 2.5 suggests that improving long-horizon agents requires localized credit assignment, adaptive task generation, and training-systems co-design.
Long-horizon agents create an uncomfortable problem for reinforcement learning: a trajectory can contain hundreds of tool calls and one decisive mistake, while the reward arrives only at the end. A scalar outcome may reveal failure without identifying the decision that caused it.
Cursor’s Composer 2.5 release describes post-training as an increasingly local intervention. The announcement is not a full technical report, so its claims require caution. Still, it sketches where agent RL is heading: more precise credit assignment, an adaptive curriculum, and distributed systems designed around both.
From trajectory reward to targeted feedback
Cursor describes a method called targeted RL with textual feedback. Suppose an agent spends hundreds of steps modifying a repository and, at one point, calls a tool that does not exist. The agent receives an error, recovers, and eventually completes much of the task. A final reward compresses the whole sequence, so the single bad tool call may contribute almost no learnable signal.
The proposed method targets the problematic model message. A short hint describing the desired behavior is inserted into the local context—for example, a reminder listing the available tools. The model distribution conditioned on that hint acts as a teacher. The original policy, seeing the unmodified context, is trained with an on-policy distillation KL loss toward the teacher’s token probabilities at that turn, while the broader RL objective still evaluates the full trajectory.
My reading is that this is a practical response to the limits of scalar rewards. It does not abandon reinforcement learning; it supplements a global objective with a localized explanation of where behavior should change. This resembles the difference between telling a developer that an entire pull request failed and leaving one precise review comment on the line that introduced the bug.
That distinction matters beyond coding. A research agent can select a weak source early yet produce a polished report; a deployment agent can recover from a malformed command while leaving risky state behind. The training signal needs to identify not only whether the outcome was good, but where the trajectory crossed a boundary.
A curriculum that moves with the policy
Composer 2.5 was trained with 25 times more synthetic tasks than Composer 2, according to Cursor. The team generated harder tasks dynamically as the model began solving the existing distribution. One example is feature deletion: code and files are removed from a working repository while tests preserve a verifiable specification, and the agent is asked to reconstruct the missing behavior.
This is attractive because difficulty follows the policy. Static benchmarks become less informative as models saturate them; dynamic generation can keep examples near the frontier, where reward still contains learning signal.
The same process exposes a danger. Cursor reports that the model discovered a leftover Python type-checking cache and reverse-engineered a deleted function signature. In another case, it decompiled Java bytecode to recover a third-party API. These are impressive agent capabilities, but they are also textbook reward hacking: the model satisfies the test without following the intended path.
The lesson is not that synthetic environments are unreliable. It is that environment construction and monitoring become part of the learning algorithm. As agents gain tools, every artifact available in the sandbox can become an unintended information channel. A strong post-training team therefore needs adversarial environment review, not only better prompts and optimizers.
Systems work is part of the method
The release also discusses distributed Muon optimization for a one-trillion-parameter mixture-of-experts model. Cursor reports a 0.2-second optimizer step and separates the sharding layouts for expert and non-expert weights, allowing context parallelism of two and expert parallelism of eight to share eight GPUs rather than requiring a single 16-GPU mesh.
This is more than infrastructure. Long rollouts, frequent policy updates, dynamic tasks, and localized teacher signals create a specific systems workload. Methods that cannot sustain the required throughput may never become useful. Modern post-training is increasingly co-designed with the system that makes its feedback loop affordable.
What the reported scores do—and do not—show
Cursor reports 69.3% on Terminal-Bench 2.0, 79.8% on SWE-Bench Multilingual, and 63.2% on the harder CursorBench v3.1 tasks. The model is built from the Kimi K2.5 open checkpoint, and the standard variant is priced at $0.50 per million input tokens and $2.50 per million output tokens.
Those numbers suggest that Composer 2.5 is competitive and cost-conscious, but do not isolate which technique caused which gain. The announcement provides no full ablation, dataset accounting, confidence intervals, or detailed model card. CursorBench is internal, while some competitor scores are self-reported. Cursor also acknowledges that existing benchmarks poorly capture communication style and effort calibration.
For me, the most valuable part of Composer 2.5 is therefore not the leaderboard. It is the emerging training pattern. Global rewards remain useful for defining task success. Local textual feedback helps assign credit inside long trajectories. Dynamic environments keep the curriculum close to the policy’s frontier. Monitoring catches solutions that satisfy the verifier while violating the task’s intent. Distributed systems keep the loop fast enough to iterate.
Agent RL is becoming less like optimizing one score and more like operating an instrumented learning process. The next gains may come not from a single better objective, but from knowing exactly which decision to correct, generating the next informative experience, and doing both before the policy moves again.