Examples
A collection of complete, runnable problems.
Rosenbrock (unconstrained)
The classic banana-shaped valley. The minimum is at $(1, 1)$ with $f = 0$.
using Retro, ForwardDiff
rosenbrock(x) = 100*(x[2] - x[1]^2)^2 + (1 - x[1])^2
prob = RetroProblem(rosenbrock, [-1.2, 1.0], AutoForwardDiff())
result = optimize(prob; display = Iteration())
@assert is_successful(result)
@assert result.fx < 1e-10Iter f(x) ||g|| Δ ρ Status
----------------------------------------------------------------------
0 2.420000e+01 2.328677e+02 1.000000e+00 0.0000 Initial
1 2.420000e+01 2.328677e+02 2.500000e-01 -0.6332 Rejected
2 6.321495e+00 6.471981e+01 2.500000e-01 0.3073 Accepted
3 3.685136e+00 2.345922e+00 5.000000e-01 0.8262 Accepted
4 3.313024e+00 2.594067e+01 5.000000e-01 0.4128 Accepted
5 2.838517e+00 2.601316e+01 5.000000e-01 0.4959 Accepted
6 2.838517e+00 2.601316e+01 1.250000e-01 -1.5116 Rejected
7 2.018627e+00 1.189224e+01 1.250000e-01 0.3365 Accepted
8 1.519271e+00 2.861394e+00 2.500000e-01 1.0690 Accepted
9 1.445345e+00 1.017989e+01 2.500000e-01 0.5229 Accepted
10 1.306137e+00 4.620038e+00 2.500000e-01 1.2130 Accepted
11 1.148998e+00 2.785608e+00 2.500000e-01 1.7237 Accepted
12 1.148998e+00 2.785608e+00 6.250000e-02 -0.1613 Rejected
13 1.018729e+00 2.462943e+00 1.250000e-01 1.0873 Accepted
14 8.268127e-01 4.485067e+00 2.500000e-01 0.9218 Accepted
15 7.159720e-01 9.298259e+00 2.500000e-01 0.7695 Accepted
16 4.987556e-01 2.119807e+00 2.500000e-01 0.9643 Accepted
17 3.711586e-01 3.128437e+00 2.500000e-01 1.5486 Accepted
18 3.711586e-01 3.128437e+00 6.250000e-02 0.0082 Rejected
19 3.228008e-01 3.837907e+00 1.250000e-01 1.2305 Accepted
20 2.146101e-01 8.496658e-01 2.500000e-01 1.0986 Accepted
21 2.146101e-01 8.496658e-01 6.250000e-02 -1.5263 Rejected
22 1.829336e-01 2.306318e+00 1.250000e-01 0.8440 Accepted
23 1.481714e-01 5.819719e+00 2.500000e-01 0.9592 Accepted
24 1.043449e-01 2.755051e+00 2.500000e-01 1.5606 Accepted
25 6.365518e-02 4.525191e+00 2.500000e-01 1.2104 Accepted
26 3.248330e-02 8.888048e-01 2.500000e-01 1.0918 Accepted
27 2.480727e-02 5.444542e+00 2.500000e-01 0.4229 Accepted
28 1.475454e-02 1.663658e-01 2.500000e-01 0.6141 Accepted
29 9.227884e-03 4.359357e-01 2.500000e-01 1.7264 Accepted
30 9.227884e-03 4.359357e-01 6.250000e-02 0.2075 Rejected
31 4.883017e-03 1.015353e+00 1.250000e-01 0.9769 Accepted
32 2.057968e-03 1.523703e+00 1.250000e-01 1.1286 Accepted
33 4.388562e-04 3.078946e-01 1.250000e-01 1.2802 Accepted
34 4.083776e-05 1.533981e-01 1.250000e-01 1.2279 Accepted
35 1.364930e-06 1.607563e-02 1.250000e-01 1.1527 Accepted
36 1.836549e-08 6.060553e-03 1.250000e-01 1.0239 Accepted
37 1.526982e-09 1.701255e-03 1.250000e-01 0.7177 Accepted
38 1.896636e-16 5.304281e-08 1.250000e-01 1.0000 Accepted
Optimization completed:
Final objective value: 1.896636e-16
Final gradient norm: 5.304281e-08
Iterations: 39
Function evaluations: 71
Gradient evaluations: 71
Termination reason: gtolRosenbrock (bounded)
Force the solution into a box.
prob = RetroProblem(rosenbrock, [-1.2, 1.0], AutoForwardDiff();
lb = [-2.0, -2.0], ub = [2.0, 2.0])
result = optimize(prob)RetroResult{Float64, Vector{Float64}}:
Final objective value: 1.8966363751127637e-16
Final gradient norm: 5.30428057190871e-8
Iterations: 39
Function evaluations: 71
Gradient evaluations: 71
Hessian evaluations: 0
Termination reason: gtol
Quadratic with narrow bounds
A simple quadratic where the solution is near a boundary.
f(x) = sum(abs2, x .- [1.0, 2.0, 3.0])
prob = RetroProblem(f, zeros(3), AutoForwardDiff();
lb = [0.5, 0.5, 0.5], ub = [1.5, 1.5, 1.5])
result = optimize(prob)
# Solution should be at [1.0, 1.5, 1.5] — clamped by upper boundsRetroResult{Float64, Vector{Float64}}:
Final objective value: 2.9952164392260188
Final gradient norm: 3.4613387232260404
Iterations: 29
Function evaluations: 33
Gradient evaluations: 33
Hessian evaluations: 0
Termination reason: ftol
Exact Hessian for small problems
prob = RetroProblem(rosenbrock, [-1.2, 1.0], AutoForwardDiff())
result = optimize(prob;
hessian_approximation = ExactHessian(),
subspace = TwoDimSubspace(),
display = Iteration())RetroResult{Float64, Vector{Float64}}:
Final objective value: 3.7455795432867295e-22
Final gradient norm: 3.382017423206331e-10
Iterations: 28
Function evaluations: 51
Gradient evaluations: 24
Hessian evaluations: 23
Termination reason: gtol
Tuning convergence tolerances
opts = RetroOptions(
gtol_a = 1e-10, # tight gradient tolerance
ftol_a = 1e-14, # tight function tolerance
initial_tr_radius = 0.1, # conservative initial radius
max_tr_radius = 100.0,
)
result = optimize(prob; options = opts, maxiter = 5000)RetroResult{Float64, Vector{Float64}}:
Final objective value: 5.3149840288005736e-21
Final gradient norm: 2.4665260431150947e-9
Iterations: 37
Function evaluations: 69
Gradient evaluations: 69
Hessian evaluations: 0
Termination reason: ftol
Trigonometric objective
trig(x) = sin(x[1])^2 + cos(x[2])^2 + x[1]*x[2]
prob = RetroProblem(trig, [1.0, -1.0], AutoForwardDiff();
lb = [-π, -π], ub = [π, π])
result = optimize(prob; display = Final())RetroResult{Float64, Vector{Float64}}:
Final objective value: -7.599924457583187
Final gradient norm: 4.712443078911963
Iterations: 1000
Function evaluations: 1987
Gradient evaluations: 1987
Hessian evaluations: 0
Termination reason: maxiter