Single Agent Patterns: Reflection and Self-Correction
Intent
Convert failure signals into improved subsequent attempts within the same episode, without weight updates.
Introduction
Even with careful planning and step-by-step logic, AI agents frequently generate incorrect code, trigger API validation failures, or encounter compilation issues. The Reflection and Self-Correction pattern offers an in-context learning mechanism to recover from these errors. By feeding detailed failure signals back to the model as verbal feedback, the agent learns to correct its mistakes dynamically in the next iteration—all without requiring expensive model weight training or fine-tuning.
Structure of the Pattern
The Reflection architecture establishes an iterative refinement cycle containing three main components:
- Generator: The primary model that receives the goal and produces a candidate output (e.g. code snippets, execution scripts, or text completions).
- Critic (Verifier / Evaluator): A component that assesses the candidate output's validity. The critic can be the same model prompted to critique its work (self-assessment), an external software verifier (such as a unit test suite, compiler, or JSON-schema parser), or direct environment errors.
- Feedback Store: If the verifier flags an error, a detailed critique is compiled (e.g., compile log, traceback, or validation messages). This feedback is appended to the agent's context window or stored in episodic memory, conditioning the Generator's next attempt.
Formalized under research paradigms like **Reflexion** [1], which models self-correction as verbal reinforcement learning, and **Self-Refine** [2], which demonstrates feedback loops on single outputs, this pattern mimics the human development cycle: write code, run tests, read exceptions, and repair the logic.
Trade-offs
Although reflection cycles are highly effective for correcting concrete bugs, they introduce specific trade-offs:
Cheap & Training-Free
Enables model adaptation during inference. There is no need for costly parameter updates, fine-tuning, or retraining pipelines.
Grounded Verification
Highly robust when critiques are anchored in deterministic outputs, such as compilation stack traces, validation logs, or unit test failures.
Illusory Self-Critique
Without external grounding (e.g. asking the model to evaluate its own textual output), correction rates drop. Models struggle to accurately judge their own generation biases.
Context Window Bloat & Instability
Repeated failures rapidly fill the context window with long exception traces. Furthermore, if a model mistakes an intermittent network glitch for a structural bug, it may rewrite working code.
Known Uses
Reflection is a key tool in software engineering agents and coding assistants:
- Reflexion Framework: Integrates dynamic verbal reinforcement and memory buffers to let agents learn from failures in reasoning and coding benchmarks [1].
- Self-Refine: Standardizes a generator-refiner-critic loop to iterate on design documents, mathematical solutions, and script creation [2].
- Verifier/Sandbox Coding Loops: Production software agents (e.g. SWE-bench solvers) that write code, execute unit tests inside isolated sandboxes, capture standard error logs, and recursively patch the code until tests pass.
References
-
[1]
Shinn, N., Labash, B., & Gopinath, D. (2023). Reflexion: Language Agents with Systematic Self-Reflection. Neural Information Processing Systems (NeurIPS).
https://arxiv.org/abs/2303.11366 -
[2]
Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Nassar, L., Goldberg, P., ... & Clark, P. (2023). Self-Refine: Iterative Refinement with Self-Feedback. Neural Information Processing Systems (NeurIPS).
https://arxiv.org/abs/2303.17651