Descent direction

In optimization, a descent direction is a vector 𝐩∈ℝn that points towards a local minimum 𝐱* of an objective function f:ℝn->ℝ. Computing 𝐱* by an iterative method, such as line search defines a descent direction 𝐩k∈ℝn at the kth iterate to be any 𝐩k such that ⟨𝐩k,∇f(𝐱k)⟩<0, where ⟨,⟩ denotes the inner product. The motivation for such an approach is that small steps along 𝐩k guarantee that f is reduced, by Taylor's theorem.

Using this definition, the negative of a non-zero gradient is always a descent direction, as ⟨−∇f(𝐱k),∇f(𝐱k)⟩=−⟨∇f(𝐱k),∇f(𝐱k)⟩<0.

Numerous methods exist to compute descent directions, all with differing merits, such as gradient descent or the conjugate gradient method.

More generally, if P is a positive definite matrix, then pk=−P∇f(xk) is a descent direction at xk.[1] This generality is used in preconditioned gradient descent methods.

## See also

* Directional derivative

## References

1. ↑ J. M. Ortega and W. C. Rheinbold (1970). Iterative Solution of Nonlinear Equations in Several Variables. pp. 243. doi:10.1137/1.9780898719468.

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Descent direction. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Descent_direction&oldid=175505"