Convex optimization

Short description: Subfield of mathematical optimization

Convex optimization is a subfield of mathematical optimization that studies the problem of minimizing convex functions over convex sets (or, equivalently, maximizing concave functions over convex sets). Many classes of convex optimization problems admit polynomial-time algorithms,[1] whereas mathematical optimization is in general NP-hard.[2][3][4]

## Contents

* 1 Definition
* 1.1 Abstract form
* 1.2 Standard form
* 1.3 Epigraph form (standard form with linear objective)
* 1.4 Conic form
* 1.5 Eliminating linear equality constraints
* 2 Special cases
* 3 Properties
* 4 Algorithms
* 4.1 Unconstrained and equality-constrained problems
* 4.2 General problems
* 5 Lagrange multipliers
* 6 Software
* 7 Applications
* 8 Extensions
* 9 See also
* 10 Notes
* 11 References
* 12 External links

## Definition

### Abstract form

A convex optimization problem is defined by two ingredients:[5][6]

* The objective function, which is a real-valued convex function of n variables, f:𝒟⊆ℝn->ℝ;
* The feasible set, which is a convex subset C⊆ℝn.

The goal of the problem is to find some 𝐱∗∈C attaining

inf⁡{f(𝐱):𝐱∈C}.

In general, there are three options regarding the existence of a solution:[7]: chpt.4

* If such a point x* exists, it is referred to as an optimal point or solution; the set of all optimal points is called the optimal set; and the problem is called solvable.
* If f is unbounded below over C, or the infimum is not attained, then the optimization problem is said to be unbounded.
* Otherwise, if C is the empty set, then the problem is said to be infeasible.

### Standard form

A convex optimization problem is in standard form if it is written as

minimize𝐱f(𝐱)subject togi(𝐱)≤0,i=1,…,mhi(𝐱)=0,i=1,…,p,

where:[7]: chpt.4

* 𝐱∈ℝn is the vector of optimization variables;
* The objective function f:𝒟⊆ℝn->ℝ is a convex function;
* The inequality constraint functions gi:ℝn->ℝ, i=1,…,m, are convex functions;
* The equality constraint functions hi:ℝn->ℝ, i=1,…,p, are affine transformations, that is, of the form: hi(𝐱)=a𝐢⋅𝐱−bi, where a𝐢 is a vector and bi is a scalar.

The feasible set C of the optimization problem consists of all points 𝐱∈𝒟 satisfying the inequality and the equality constraints. This set is convex because 𝒟 is convex, the sublevel sets of convex functions are convex, affine sets are convex, and the intersection of convex sets is convex.[7]: chpt.2

Many optimization problems can be equivalently formulated in this standard form. For example, the problem of maximizing a concave function f can be re-formulated equivalently as the problem of minimizing the convex function −f. The problem of maximizing a concave function over a convex set is commonly called a convex optimization problem.[8]

### Epigraph form (standard form with linear objective)

In the standard form it is possible to assume, without loss of generality, that the objective function f is a linear function. This is because any program with a general objective can be transformed into a program with a linear objective by adding a single variable t and a single constraint, as follows:[9]: 1.4

minimize𝐱,ttsubject tof(𝐱)−t≤0gi(𝐱)≤0,i=1,…,mhi(𝐱)=0,i=1,…,p,

### Conic form

Every convex program can be presented in a conic form, which means minimizing a linear objective over the intersection of an affine plane and a convex cone:[9]: 5.1

minimize𝐱cTxsubject tox∈(b+L)∩K

where K is a closed pointed convex cone, L is a linear subspace of Rn, and b is a vector in Rn. A linear program in standard form is the special case in which K is the nonnegative orthant of Rn.

### Eliminating linear equality constraints

It is possible to convert a convex program in standard form, to a convex program with no equality constraints.[7]: 132 Denote the equality constraints hi(x)=0 as Ax=b, where A has n columns. If Ax=b is infeasible, then of course the original problem is infeasible. Otherwise, it has some solution x0 , and the set of all solutions can be presented as: Fz+x0, where z is in Rk, k=n-rank(A), and F is an n-by-k matrix. Substituting x = Fz+x0 in the original problem gives:

> minimize𝐱f(𝐅𝐳+𝐱𝟎)subject togi(𝐅𝐳+𝐱𝟎)≤0,i=1,…,m

where the variables are z. Note that there are rank(A) fewer variables. This means that, in principle, one can restrict attention to convex optimization problems without equality constraints. In practice, however, it is often preferred to retain the equality constraints, since they might make some algorithms more efficient, and also make the problem easier to understand and analyze.

## Special cases

The following problem classes are all convex optimization problems, or can be reduced to convex optimization problems via simple transformations:[7]: chpt.4 [10]

A hierarchy of convex optimization problems. (LP: linear programming, QP: quadratic programming, SOCP second-order cone program, SDP: semidefinite programming, CP: conic optimization.)

* Linear programming problems are the simplest convex programs. In LP, the objective and constraint functions are all linear.
* Quadratic programming are the next-simplest. In QP, the constraints are all linear, but the objective may be a convex quadratic function.
* Second order cone programming are more general.
* Semidefinite programming are more general.
* Conic optimization are even more general - see figure to the right,

Other special cases include;

* Least squares
* Quadratic minimization with convex quadratic constraints
* Geometric programming
* Entropy maximization with appropriate constraints.

## Properties

The following are useful properties of convex optimization problems:[11][7]: chpt.4

* every point that is local minimum is also a global minimum;
* the optimal set is convex;
* if the objective function is strictly convex, then the problem has at most one optimal point.

## Algorithms

### Unconstrained and equality-constrained problems

The convex programs easiest to solve are the unconstrained problems, or the problems with only equality constraints. As the equality constraints are all linear, they can be eliminated with linear algebra and integrated into the objective, thus converting an equality-constrained problem into an unconstrained one.

In the class of unconstrained (or equality-constrained) problems, the simplest ones are those in which the objective is quadratic. For these problems, the KKT conditions (which are necessary for optimality) are all linear, so they can be solved analytically.[7]: chpt.11

For unconstrained (or equality-constrained) problems with a general convex objective that is twice-differentiable, Newton's method can be used. It can be seen as reducing a general unconstrained convex problem, to a sequence of quadratic problems.[7]: chpt.11 Newton's method can be combined with line search for an appropriate step size, and it can be mathematically proven to converge quickly.

Other efficient algorithms for unconstrained minimization are gradient descent (a special case of steepest descent).

### General problems

The more challenging problems are those with inequality constraints. A common way to solve them is to reduce them to unconstrained problems by adding a barrier function, enforcing the inequality constraints, to the objective function. Such methods are called interior point methods.[7]: chpt.11 They have to be initialized by finding a feasible interior point using by so-called phase I methods, which either find a feasible point or show that none exist. Phase I methods generally consist of reducing the search in question to a simpler convex optimization problem.[7]: chpt.11

Convex optimization problems can also be solved by the following contemporary methods:[12]

* Bundle methods (Wolfe, Lemaréchal, Kiwiel), and
* Subgradient projection methods (Polyak),
* Interior-point methods,[1] which make use of self-concordant barrier functions [13] and self-regular barrier functions.[14]
* Cutting-plane methods
* Ellipsoid method
* Subgradient method
* Dual subgradients and the drift-plus-penalty method

Subgradient methods can be implemented simply and so are widely used.[15] Dual subgradient methods are subgradient methods applied to a dual problem. The drift-plus-penalty method is similar to the dual subgradient method, but takes a time average of the primal variables.

## Lagrange multipliers

Main article: Lagrange multiplier

Consider a convex minimization problem given in standard form by a cost function f(x) and inequality constraints gi(x)≤0 for 1≤i≤m. Then the domain 𝒳 is:

𝒳={x∈X|g1(x),…,gm(x)≤0}.

The Lagrangian function for the problem is[16]

L(x,λ0,λ1,…,λm)=λ0f(x)+λ1g1(x)+⋯+λmgm(x).

For each point x in X that minimizes f over X, there exist real numbers λ0,λ1,…,λm, called Lagrange multipliers, that satisfy these conditions simultaneously:

1. x minimizes L(y,λ0,λ1,…,λm) over all y∈X,
2. λ0,λ1,…,λm≥0, with at least one λk>0,
3. λ1g1(x)=⋯=λmgm(x)=0 (complementary slackness).

If there exists a "strictly feasible point", that is, a point z satisfying

g1(z),…,gm(z)<0,

then the statement above can be strengthened to require that λ0=1.

Conversely, if some x in X satisfies (1)–(3) for scalars λ0,…,λm with λ0=1 then x is certain to minimize f over X.

## Software

There is a large software ecosystem for convex optimization. This ecosystem has two main categories: solvers on the one hand and modeling tools (or interfaces) on the other hand.

Solvers implement the algorithms themselves and are usually written in C. They require users to specify optimization problems in very specific formats which may not be natural from a modeling perspective. Modeling tools are separate pieces of software that let the user specify an optimization in higher-level syntax. They manage all transformations to and from the user's high-level model and the solver's input/output format.

Below are two tables. The first shows modelling tools (such as CVXPY and JuMP.jl) and the second shows solvers (such as SCS and MOSEK). They are by no means exhaustive.

Program  | Language  | Description  | FOSS?  | Ref.

|
|
|
|

CVX  | MATLAB | Interfaces with SeDuMi and SDPT3 solvers; designed to only express convex optimization problems.  | Yes  | [17]
CVXPY  | Python  |  | Yes  | [18]
Convex.jl  | Julia | Disciplined convex programming, supports many solvers.  | Yes  | [19]
CVXR  | R |  | Yes  | [20]
GAMS  |  | Modeling system for linear, nonlinear, mixed integer linear/nonlinear, and second-order cone programming problems.  | No  | [17]
GloptiPoly  | MATLAB,

Octave

| Modeling system for polynomial optimization.  | Yes  | [17]
JuMP.jl  | Julia | Supports many solvers. Also supports integer and nonlinear optimization, and some nonconvex optimization.  | Yes  | [21]
ROME  |  | Modeling system for robust optimization. Supports distributionally robust optimization and uncertainty sets.  | Yes  | [17]
SOSTOOLS  |  | Modeling system for polynomial optimization. Uses SDPT3 and SeDuMi. Requires Symbolic Computation Toolbox.  | Yes  | [17]
SparsePOP  |  | Modeling system for polynomial optimization. Uses the SDPA or SeDuMi solvers.  | Yes  | [17]
YALMIP  | MATLAB, Octave  | Interfaces with CPLEX, GUROBI, MOSEK, SDPT3, SEDUMI, CSDP, SDPA, PENNON solvers; also supports integer and nonlinear optimization, and some nonconvex optimization. Can perform robust optimization with uncertainty in LP/SOCP/SDP constraints.  | Yes  | [17]
Program  | Language  | Description  | FOSS?  | Ref.

|
|
|
|

AIMMS  |  | Can do robust optimization on linear programming (with MOSEK to solve second-order cone programming) and mixed integer linear programming. Modeling package for LP + SDP and robust versions.  | No  | [17]
CPLEX  |  | Supports primal-dual methods for LP + SOCP. Can solve LP, QP, SOCP, and mixed integer linear programming problems.  | No  | [17]
CSDP  | C | Supports primal-dual methods for LP + SDP. Interfaces available for MATLAB, R, and Python. Parallel version available. SDP solver.  | Yes  | [17]
CVXOPT | Python  | Supports primal-dual methods for LP + SOCP + SDP. Uses Nesterov-Todd scaling. Interfaces to MOSEK and DSDP.  | Yes  | [17]
MOSEK  |  | Supports primal-dual methods for LP + SOCP.  | No  | [17]
SeDuMi  | MATLAB, Octave, MEX | Solves LP + SOCP + SDP. Supports primal-dual methods for LP + SOCP + SDP.  | Yes  | [17]
SDPA  | C++ | Solves LP + SDP. Supports primal-dual methods for LP + SDP. Parallelized and extended precision versions are available.  | Yes  | [17]
SDPT3  | MATLAB, Octave, MEX  | Solves LP + SOCP + SDP. Supports primal-dual methods for LP + SOCP + SDP.  | Yes  | [17]
ConicBundle  |  | Supports general-purpose codes for LP + SOCP + SDP. Uses a bundle method. Special support for SDP and SOCP constraints.  | Yes  | [17]
DSDP  |  | Supports general-purpose codes for LP + SDP. Uses a dual interior point method.  | Yes  | [17]
LOQO  |  | Supports general-purpose codes for SOCP, which it treats as a nonlinear programming problem.  | No  | [17]
PENNON  |  | Supports general-purpose codes. Uses an augmented Lagrangian method, especially for problems with SDP constraints.  | No  | [17]
SDPLR  |  | Supports general-purpose codes. Uses low-rank factorization with an augmented Lagrangian method.  | Yes  | [17]

## Applications

Convex optimization can be used to model problems in a wide range of disciplines, such as automatic control systems, estimation and signal processing, communications and networks, electronic circuit design,[7]: 17 data analysis and modeling, finance, statistics (optimal experimental design),[22] and structural optimization, where the approximation concept has proven to be efficient.[7][23] Convex optimization can be used to model problems in the following fields:

* Portfolio optimization.[24]
* Worst-case risk analysis.[24]
* Optimal advertising.[24]
* Variations of statistical regression (including regularization and quantile regression).[24]
* Model fitting[24] (particularly multiclass classification[25]).
* Electricity generation optimization.[25]
* Combinatorial optimization.[25]
* Non-probabilistic modelling of uncertainty.[26]
* Localization using wireless signals [27]

## Extensions

## See also

* Duality
* Karush–Kuhn–Tucker conditions
* Optimization problem
* Proximal gradient method
* Algorithmic problems on convex sets

## Notes

1. ↑ 1.0 1.1 Nesterov & Nemirovskii 1994
2. ↑ Murty, Katta; Kabadi, Santosh (1987). "Some NP-complete problems in quadratic and nonlinear programming". Mathematical Programming 39 (2): 117–129. doi:10.1007/BF02592948. Bibcode: 1987MatPr..39..117M.
3. ↑ Sahni, S. "Computationally related problems," in SIAM Journal on Computing, 3, 262--279, 1974.
4. ↑ Pardalos, Panos M.; Vavasis, Stephen A. (1991). "Quadratic programming with one negative eigenvalue is NP-hard". Journal of Global Optimization 1: 15–22. doi:10.1007/BF00120662. https://link.springer.com/article/10.1007/BF00120662.
5. ↑ Hiriart-Urruty, Jean-Baptiste; Lemaréchal, Claude (1996). Convex analysis and minimization algorithms: Fundamentals. Springer. p. 291. ISBN 9783540568506. https://books.google.com/books?id=Gdl4Jc3RVjcC&q=lemarechal+convex+analysis+and+minimization.
6. ↑ Ben-Tal, Aharon; Nemirovskiĭ, Arkadiĭ Semenovich (2001). Lectures on modern convex optimization: analysis, algorithms, and engineering applications. pp. 335–336. ISBN 9780898714913. https://books.google.com/books?id=M3MqpEJ3jzQC&q=Lectures+on+Modern+Convex+Optimization:+Analysis,+Algorithms,.
7. ↑ 7.00 7.01 7.02 7.03 7.04 7.05 7.06 7.07 7.08 7.09 7.10 7.11 Boyd, Stephen; Vandenberghe, Lieven (2004). Convex Optimization. Cambridge University Press. ISBN 978-0-521-83378-3. https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf. Retrieved 12 Apr 2021.
8. ↑ "Optimization Problem Types - Convex Optimization". 9 January 2011. https://www.solver.com/convex-optimization.
9. ↑ 9.0 9.1 Arkadi Nemirovsky (2004). Interior point polynomial-time methods in convex programming. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=8c3cb6395a35cb504019f87f447d65cb6cf1cdf0.
10. ↑ Agrawal, Akshay; Verschueren, Robin; Diamond, Steven; Boyd, Stephen (2018). "A rewriting system for convex optimization problems". Control and Decision 5 (1): 42–60. doi:10.1080/23307706.2017.1397554. https://web.stanford.edu/~boyd/papers/pdf/cvxpy_rewriting.pdf.
11. ↑ Rockafellar, R. Tyrrell (1993). "Lagrange multipliers and optimality". SIAM Review 35 (2): 183–238. doi:10.1137/1035044. Bibcode: 1993SIAMR..35..183R. http://web.williams.edu/Mathematics/sjmiller/public_html/105Sp10/handouts/Rockafellar_LagrangeMultAndOptimality.pdf.
12. ↑ For methods for convex minimization, see the volumes by Hiriart-Urruty and Lemaréchal (bundle) and the textbooks by Ruszczyński, Bertsekas, and Boyd and Vandenberghe (interior point).
13. ↑ Nesterov, Yurii; Arkadii, Nemirovskii (1995). Interior-Point Polynomial Algorithms in Convex Programming. Society for Industrial and Applied Mathematics. ISBN 978-0898715156.
14. ↑ Peng, Jiming; Roos, Cornelis; Terlaky, Tamás (2002). "Self-regular functions and new search directions for linear and semidefinite optimization". Mathematical Programming 93 (1): 129–171. doi:10.1007/s101070200296. ISSN 0025-5610.
15. ↑ "Numerical Optimization" (in en). Springer Series in Operations Research and Financial Engineering. 2006. doi:10.1007/978-0-387-40065-5. ISBN 978-0-387-30303-1. https://link.springer.com/book/10.1007/978-0-387-40065-5.
16. ↑ Beavis, Brian; Dobbs, Ian M. (1990). "Static Optimization". Optimization and Stability Theory for Economic Analysis. New York: Cambridge University Press. p. 40. ISBN 0-521-33605-8. https://books.google.com/books?id=L7HMACFgnXMC&pg=PA40.
17. ↑ 17.00 17.01 17.02 17.03 17.04 17.05 17.06 17.07 17.08 17.09 17.10 17.11 17.12 17.13 17.14 17.15 17.16 17.17 17.18 17.19 Borchers, Brian. "An Overview Of Software For Convex Optimization". http://infohost.nmt.edu/~borchers/presentation.pdf.
18. ↑ "Welcome to CVXPY 1.1 — CVXPY 1.1.11 documentation". https://www.cvxpy.org/.
19. ↑ Udell, Madeleine; Mohan, Karanveer; Zeng, David; Hong, Jenny; Diamond, Steven; Boyd, Stephen (2014-10-17). "Convex Optimization in Julia". arXiv:1410.4821 [math.OC].
20. ↑ "Disciplined Convex Optimiation - CVXR". https://www.cvxgrp.org/CVXR/.
21. ↑ Lubin, Miles; Dowson, Oscar; Dias Garcia, Joaquim; Huchette, Joey; Legat, Benoît; Vielma, Juan Pablo (2023). "JuMP 1.0: Recent improvements to a modeling language for mathematical optimization". Mathematical Programming Computation 15 (3): 581–589. doi:10.1007/s12532-023-00239-3.
22. ↑ Christensen/Klarbring, chpt. 4.
23. ↑ Schmit, L.A.; Fleury, C. 1980: Structural synthesis by combining approximation concepts and dual methods. J. Amer. Inst. Aeronaut. Astronaut 18, 1252-1260
24. ↑ 24.0 24.1 24.2 24.3 24.4 Boyd, Stephen; Diamond, Stephen; Zhang, Junzi; Agrawal, Akshay. "Convex Optimization Applications". https://web.stanford.edu/~boyd/papers/pdf/cvx_applications.pdf.
25. ↑ 25.0 25.1 25.2 Malick, Jérôme (2011-09-28). "Convex optimization: applications, formulations, relaxations". https://www-ljk.imag.fr//membres/Jerome.Malick/Talks/11-INRIA.pdf.
26. ↑ Ben Haim Y. and Elishakoff I., Convex Models of Uncertainty in Applied Mechanics, Elsevier Science Publishers, Amsterdam, 1990
27. ↑ Ahmad Bazzi, Dirk TM Slock, and Lisa Meilhac. "Online angle of arrival estimation in the presence of mutual coupling." 2016 IEEE Statistical Signal Processing Workshop (SSP). IEEE, 2016.

## References

* Bertsekas, Dimitri P.; Nedic, Angelia; Ozdaglar, Asuman (2003). Convex Analysis and Optimization. Belmont, MA.: Athena Scientific. ISBN 978-1-886529-45-8.
* Bertsekas, Dimitri P. (2009). Convex Optimization Theory. Belmont, MA.: Athena Scientific. ISBN 978-1-886529-31-1.
* Bertsekas, Dimitri P. (2015). Convex Optimization Algorithms. Belmont, MA.: Athena Scientific. ISBN 978-1-886529-28-1.
* Borwein, Jonathan; Lewis, Adrian (2000). Convex Analysis and Nonlinear Optimization: Theory and Examples, Second Edition. Springer. https://carma.newcastle.edu.au/resources/jon/Preprints/Books/CaNo2/cano2f.pdf. Retrieved 12 Apr 2021.
* Christensen, Peter W.; Anders Klarbring (2008). An introduction to structural optimization. 153. Springer Science & Business Media. ISBN 9781402086663. https://books.google.com/books?id=80IeN__MYI8C.

* Hiriart-Urruty, Jean-Baptiste, and Lemaréchal, Claude. (2004). Fundamentals of Convex analysis. Berlin: Springer.
* Hiriart-Urruty, Jean-Baptiste; Lemaréchal, Claude (1993). Convex analysis and minimization algorithms, Volume I: Fundamentals. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences]. 305. Berlin: Springer-Verlag. pp. xviii+417. ISBN 978-3-540-56850-6.
* Hiriart-Urruty, Jean-Baptiste; Lemaréchal, Claude (1993). Convex analysis and minimization algorithms, Volume II: Advanced theory and bundle methods. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences]. 306. Berlin: Springer-Verlag. pp. xviii+346. ISBN 978-3-540-56852-0.
* Kiwiel, Krzysztof C. (1985). Methods of Descent for Nondifferentiable Optimization. Lecture Notes in Mathematics. New York: Springer-Verlag. ISBN 978-3-540-15642-0. https://archive.org/details/methodsofdescent0000kiwi.
* Lemaréchal, Claude (2001). "Lagrangian relaxation". in Michael Jünger and Denis Naddef. Computational combinatorial optimization: Papers from the Spring School held in Schloß Dagstuhl, May 15–19, 2000. Lecture Notes in Computer Science. 2241. Berlin: Springer-Verlag. pp. 112–156. doi:10.1007/3-540-45586-8_4. ISBN 978-3-540-42877-0.
* Nesterov, Yurii; Nemirovskii, Arkadii (1994). Interior Point Polynomial Methods in Convex Programming. SIAM.
* Nesterov, Yurii. (2004). Introductory Lectures on Convex Optimization, Kluwer Academic Publishers
* Rockafellar, R. T. (1970). Convex analysis. Princeton: Princeton University Press.

* Ruszczyński, Andrzej (2006). Nonlinear Optimization. Princeton University Press.
* Schmit, L.A.; Fleury, C. 1980: Structural synthesis by combining approximation concepts and dual methods. J. Amer. Inst. Aeronaut. Astronaut 18, 1252-1260

## External links

* EE364a: Convex Optimization I and EE364b: Convex Optimization II, Stanford course homepages
* 6.253: Convex Analysis and Optimization, an MIT OCW course homepage
* Brian Borchers, An overview of software for convex optimization
* Convex Optimization Book by Lieven Vandenberghe and Stephen P. Boyd

* v
* t
* e

Optimization: Algorithms, methods, and heuristics

Unconstrained nonlinear

| … functions|

* Golden-section search
* Interpolation methods
* Line search
* Nelder–Mead method
* Successive parabolic interpolation

|

… and gradients|

| Convergence|

* Trust region
* Wolfe conditions

|

Quasi–Newton|

* Berndt–Hall–Hall–Hausman
* Broyden–Fletcher–Goldfarb–Shanno and L-BFGS
* Davidon–Fletcher–Powell
* Symmetric rank-one (SR1)

Other methods|

* Gauss–Newton
* Gradient
* Levenberg–Marquardt
* Conjugate gradient
* Truncated Newton

… and Hessians|

* Newton's method

Optimization computes maxima and minima.

Constrained nonlinear

| General|

* Barrier methods
* Penalty methods

|

Differentiable|

* Augmented Lagrangian methods
* Sequential quadratic programming
* Successive linear programming

Convex optimization

| Convex
minimization|

* Cutting-plane method
* Reduced gradient (Frank–Wolfe)
* Subgradient method

|

Linear and
quadratic|

| Interior point|

* Affine scaling
* Ellipsoid algorithm of Khachiyan
* Projective algorithm of Karmarkar

|

Basis-exchange|

* Simplex algorithm of Dantzig
* Revised simplex algorithm
* Criss-cross algorithm
* Principal pivoting algorithm of Lemke

Combinatorial

| Paradigms|

* Approximation algorithm
* Dynamic programming
* Greedy algorithm
* Integer programming
* Branch and bound/cut

|

Graph
algorithms|

| Minimum
spanning tree|

* Borůvka
* Prim
* Kruskal

|

* Bellman–Ford
* Dijkstra
* Floyd–Warshall

Network flows|

* Dinic
* Edmonds–Karp
* Ford–Fulkerson
* Push–relabel maximum flow

Metaheuristics

* Evolutionary algorithm
* Hill climbing
* Local search
* Simulated annealing
* Tabu search

* Software

Template:Convex analysis and variational analysis

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Convex optimization. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Convex_optimization&oldid=4299586"
*[Ref.]: Reference(s)
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template