# Optim.jl

Univariate and multivariate optimization in Julia.

Optim.jl is part of the JuliaNLSolvers family.

## Help and support

For help and support, please post on the Optimization (Mathematical) section of the Julia discourse or the #math-optimization channel of the Julia slack.

## Installation

Install Optim.jl using the Julia package manager:

import Pkg
Pkg.add("Optim")

## Documentation

The online documentation is available at https://julianlsolvers.github.io/Optim.jl/stable.

## Example

To minimize the Rosenbrock function, do:

julia> using Optim

julia> rosenbrock(x) =  (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2
rosenbrock (generic function with 1 method)

julia> result = optimize(rosenbrock, zeros(2), BFGS())
* Status: success

* Candidate solution
Final objective value:     5.471433e-17

* Found with
Algorithm:     BFGS

* Convergence measures
|x - x'|               = 3.47e-07 ≰ 0.0e+00
|x - x'|/|x'|          = 3.47e-07 ≰ 0.0e+00
|f(x) - f(x')|         = 6.59e-14 ≰ 0.0e+00
|f(x) - f(x')|/|f(x')| = 1.20e+03 ≰ 0.0e+00
|g(x)|                 = 2.33e-09 ≤ 1.0e-08

* Work counters
Seconds run:   0  (vs limit Inf)
Iterations:    16
f(x) calls:    53
∇f(x) calls:   53

julia> Optim.minimizer(result)
2-element Vector{Float64}:
0.9999999926033423
0.9999999852005355

julia> Optim.minimum(result)
5.471432670590216e-17

To get information on the keywords used to construct method instances, use the Julia REPL help prompt (?)

help?> LBFGS
search: LBFGS

LBFGS
≡≡≡≡≡

Constructor
===========

LBFGS(; m::Integer = 10,
alphaguess = LineSearches.InitialStatic(),
linesearch = LineSearches.HagerZhang(),
P=nothing,
precondprep = (P, x) -> nothing,
manifold = Flat(),
scaleinvH0::Bool = true && (typeof(P) <: Nothing))

LBFGS has two special keywords; the memory length m, and the scaleinvH0 flag.
The memory length determines how many previous Hessian approximations to
store. When scaleinvH0 == true, then the initial guess in the two-loop
recursion to approximate the inverse Hessian is the scaled identity, as can be
found in Nocedal and Wright (2nd edition) (sec. 7.2).

In addition, LBFGS supports preconditioning via the P and precondprep keywords.

Description
===========

The LBFGS method implements the limited-memory BFGS algorithm as described in
Nocedal and Wright (sec. 7.2, 2006) and original paper by Liu & Nocedal
(1989). It is a quasi-Newton method that updates an approximation to the
Hessian using past approximations as well as the gradient.

References
==========

•  Wright, S. J. and J. Nocedal (2006), Numerical optimization, 2nd edition.
Springer

•  Liu, D. C. and Nocedal, J. (1989). "On the Limited Memory Method for
Large Scale Optimization". Mathematical Programming B. 45 (3): 503–528

## Use with JuMP

You can use Optim.jl with JuMP.jl as follows:

julia> using JuMP, Optim

julia> model = Model(Optim.Optimizer);

julia> set_optimizer_attribute(model, "method", BFGS())

julia> @variable(model, x[1:2]);

julia> @objective(model, Min, (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2)
(x[1]² - 2 x[1] + 1) + (100.0 * ((-x[1]² + x[2]) ^ 2.0))

julia> optimize!(model)

julia> objective_value(model)
3.7218241804173566e-21

julia> value.(x)
2-element Vector{Float64}:
0.9999999999373603
0.99999999986862

## Citation

If you use Optim.jl in your work, please cite the following:

@article{mogensen2018optim,
author  = {Mogensen, Patrick Kofod and Riseth, Asbj{\o}rn Nilsen},
title   = {Optim: A mathematical optimization package for {Julia}},
journal = {Journal of Open Source Software},
year    = {2018},
volume  = {3},
number  = {24},
pages   = {615},
doi     = {10.21105/joss.00615}
}