Proximal operator

Short description: Function in mathematical optimization

In mathematical optimization, the proximal operator is an operator associated with a proper,[note 1] lower semi-continuous convex function f from a Hilbert space 𝒳 to [−∞,+∞], and is defined by:[1]

proxf(v)=arg⁡minx∈𝒳(f(x)+12‖x−v‖𝒳2).

For any function in this class, the minimizer of the right-hand side above is unique, hence making the proximal operator well-defined. The proximal operator is used in proximal gradient methods, which is frequently used in optimization algorithms associated with non-differentiable optimization problems such as total variation denoising.

## Contents

* 1 Properties
* 2 Notes
* 3 References
* 4 See also
* 5 External links

## Properties

The prox of a proper, lower semi-continuous convex function f enjoys several useful properties for optimization.

* Fixed points of proxf are minimizers of f: {x∈𝒳 | proxfx=x}=arg⁡min⁡f.
* Global convergence to a minimizer is defined as follows: If arg⁡min⁡f≠∅, then for any initial point x0∈𝒳, the recursion (∀n∈ℕ)xn+1=proxfxn yields convergence xn->x∈arg⁡min⁡f as n->+∞. This convergence may be weak if 𝒳 is infinite dimensional.[2]
* The proximal operator can be seen as a generalization of the projection operator. Indeed, in the specific case where f is the 0-∞ characteristic function ιC of a nonempty, closed, convex set C we have that

proxιC(x)=argminy{12‖x−y‖22if y∈C+∞if y∉C=argminy∈C12‖x−y‖22
showing that the proximity operator is indeed a generalisation of the projection operator.

* A function is firmly non-expansive if (∀(x,y)∈𝒳2)‖proxfx−proxfy‖2≤⟨x−y ,proxfx−proxfy⟩.
* The proximal operator of a function is related to the gradient of the Moreau envelope Mλf of a function λf by the following identity: ∇Mλf(x)=1λ(x−proxλf(x)).
* The proximity operator of f is characterized by inclusion p=proxf(x)⇔x−p∈∂f(p), where ∂f is the subdifferential of f, given by

∂f(x)={u∈ℝN∣∀y∈ℝN,(y−x)Tu+f(x)≤f(y)} In particular, If f is differentiable then the above equation reduces to p=proxf(x)⇔x−p=∇f(p).

## Notes

1. ↑ An (extended) real-valued function f on a Hilbert space is said to be proper if it is not identically equal to +∞, and −∞ is not in its image.

## References

1. ↑ Neal Parikh and Stephen Boyd (2013). "Proximal Algorithms". Foundations and Trends in Optimization 1 (3): 123–231. https://web.stanford.edu/~boyd/papers/pdf/prox_algs.pdf. Retrieved 2019-01-29.
2. ↑ Bauschke, Heinz H.; Combettes, Patrick L. (2017). Convex Analysis and Monotone Operator Theory in Hilbert Spaces. CMS Books in Mathematics. New York: Springer. doi:10.1007/978-3-319-48311-5. ISBN 978-3-319-48310-8.

## See also

* Proximal gradient method

## External links

* The Proximity Operator repository: a collection of proximity operators implemented in Matlab and Python.
* ProximalOperators.jl: a Julia package implementing proximal operators.
* ODL: a Python library for inverse problems that utilizes proximal operators.

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Proximal operator. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Proximal_operator&oldid=4154495"