Algorithm Tutorial — Overview
This tutorial builds the Retro optimizer one concept at a time, starting from the simplest possible optimizer and ending with a production-grade trust-region reflective method.
Each page answers three questions:
- What problem are we trying to solve?
- What idea fixes it?
- How does Retro implement it?
You do not need to read these pages to use Retro — the Quick Start is enough for that. This tutorial is for anyone who wants to understand what happens under the hood.
The chapters
| # | Page | One-sentence summary |
|---|---|---|
| 1 | The Simplest Optimizer | Gradient descent: easy, but unreliable. |
| 2 | Adding Trust Regions | A safety radius that adapts to local curvature. |
| 3 | Handling Bounds | Reflecting steps at box boundaries à la Coleman–Li. |
| 4 | Working in Subspaces | Solving the TR problem cheaply in 2-D or via CG. |
| 5 | Hessian Approximations | BFGS, SR1, and exact Hessians — trade-offs. |
| 6 | Robustness & Fallbacks | What happens when things go wrong. |
Start with Chapter 1, or jump to any topic you are curious about.