Optimization and root finding (scipy.optimize) — SciPy v1.16.2 Manual (original) (raw)

SciPy optimize provides functions for minimizing (or maximizing) objective functions, possibly subject to constraints. It includes solvers for nonlinear problems (with support for both local and global optimization algorithms), linear programming, constrained and nonlinear least-squares, root finding, and curve fitting.

Common functions and objects, shared across different solvers, are:

Optimization#

Scalar functions optimization#

The minimize_scalar function supports the following methods:

• minimize_scalar(method=’brent’)
• minimize_scalar(method=’bounded’)
• minimize_scalar(method=’golden’)

Local (multivariate) optimization#

The minimize function supports the following methods:

• minimize(method=’Nelder-Mead’)
• minimize(method=’Powell’)
• minimize(method=’CG’)
• minimize(method=’BFGS’)
• minimize(method=’Newton-CG’)
• minimize(method=’L-BFGS-B’)
• minimize(method=’TNC’)
• minimize(method=’COBYLA’)
• minimize(method=’COBYQA’)
• minimize(method=’SLSQP’)
• minimize(method=’trust-constr’)
• minimize(method=’dogleg’)
• minimize(method=’trust-ncg’)
• minimize(method=’trust-krylov’)
• minimize(method=’trust-exact’)

Constraints are passed to minimize function as a single object or as a list of objects from the following classes:

Simple bound constraints are handled separately and there is a special class for them:

Quasi-Newton strategies implementing HessianUpdateStrategyinterface can be used to approximate the Hessian in minimizefunction (available only for the ‘trust-constr’ method). Available quasi-Newton methods implementing this interface are:

Global optimization#

Least-squares and curve fitting#

Nonlinear least-squares#

Linear least-squares#

Curve fitting#

Root finding#

Scalar functions#

The root_scalar function supports the following methods:

• root_scalar(method=’brentq’)
• root_scalar(method=’brenth’)
• root_scalar(method=’bisect’)
• root_scalar(method=’ridder’)
• root_scalar(method=’newton’)
• root_scalar(method=’toms748’)
• root_scalar(method=’secant’)
• root_scalar(method=’halley’)

The table below lists situations and appropriate methods, along with_asymptotic_ convergence rates per iteration (and per function evaluation) for successful convergence to a simple root(*). Bisection is the slowest of them all, adding one bit of accuracy for each function evaluation, but is guaranteed to converge. The other bracketing methods all (eventually) increase the number of accurate bits by about 50% for every function evaluation. The derivative-based methods, all built on newton, can converge quite quickly if the initial value is close to the root. They can also be applied to functions defined on (a subset of) the complex plane.

Fixed point finding:

Multidimensional#

The root function supports the following methods:

• root(method=’hybr’)
• root(method=’lm’)
• root(method=’broyden1’)
• root(method=’broyden2’)
• root(method=’anderson’)
• root(method=’linearmixing’)
• root(method=’diagbroyden’)
• root(method=’excitingmixing’)
• root(method=’krylov’)
• root(method=’df-sane’)

Elementwise Minimization and Root Finding#

• Elementwise Scalar Optimization (scipy.optimize.elementwise)
  • Root finding
    * find_root
    * bracket_root
  • Minimization
    * find_minimum
    * bracket_minimum

Linear programming / MILP#

The linprog function supports the following methods:

• linprog(method=’simplex’)
• linprog(method=’interior-point’)
• linprog(method=’revised simplex’)
• linprog(method=’highs-ipm’)
• linprog(method=’highs-ds’)
• linprog(method=’highs’)

The simplex, interior-point, and revised simplex methods support callback functions, such as:

Assignment problems#

The quadratic_assignment function supports the following methods:

• quadratic_assignment(method=’faq’)
• quadratic_assignment(method=’2opt’)

Utilities#

Finite-difference approximation#

Line search#

Hessian approximation#

Benchmark problems#

Legacy functions#

The functions below are not recommended for use in new scripts; all of these methods are accessible via a newer, more consistent interfaces, provided by the interfaces above.

Optimization#

General-purpose multivariate methods:

Constrained multivariate methods:

Univariate (scalar) minimization methods:

Least-squares#

Root finding#

General nonlinear solvers:

Large-scale nonlinear solvers:

Simple iteration solvers: