Moreau envelope

Short description: Mathematical optimization function

The Moreau envelope (or the Moreau-Yosida regularization) Mf of a proper lower semi-continuous convex function f is a smoothed version of f. It was proposed by Jean-Jacques Moreau in 1965.[1]

The Moreau envelope has important applications in mathematical optimization: minimizing over Mf and minimizing over f are equivalent problems in the sense that the sets of minimizers of f and Mf are the same. However, first-order optimization algorithms can be directly applied to Mf, since f may be non-differentiable while Mf is always continuously differentiable. Indeed, many proximal gradient methods can be interpreted as a gradient descent method over Mf.

## Contents

* 1 Definition
* 2 Properties
* 3 See also
* 4 References
* 5 External links

## Definition

The Moreau envelope of a proper lower semi-continuous convex function f from a Hilbert space 𝒳 to (−∞,+∞] is defined as[2]

Mf(v)=infx∈𝒳(f(x)+12‖x−v‖22).

Given a parameter λ∈ℝ, the Moreau envelope of λf is also called as the Moreau envelope of f with parameter λ.[2]

## Properties

* The Moreau envelope can also be seen as the infimal convolution between f and (1/2)‖⋅‖22.
* The proximal operator of a function is related to the gradient of the Moreau envelope by the following identity:

∇Mλf(x)=1λ(x−proxλf(x)). By defining the sequence xk+1=proxλf(xk) and using the above identity, we can interpret the proximal operator as a gradient descent algorithm over the Moreau envelope.

* Using Fenchel's duality theorem, one can derive the following dual formulation of the Moreau envelope:

Mλf(v)=maxp∈𝒳(⟨p,v⟩−λ2‖p‖2−f*(p)), where f* denotes the convex conjugate of f. Since the subdifferential of a proper, convex, lower semicontinuous function on a Hilbert space is inverse to the subdifferential of its convex conjugate, we can conclude that if p0∈𝒳 is the maximizer of the above expression, then x0:=v−λp0 is the minimizer in the primal formulation and vice versa.

* By Hopf–Lax formula, the Moreau envelope is a viscosity solution to a Hamilton–Jacobi equation.[3] Stanley Osher and co-authors used this property and Cole–Hopf transformation to derive an algorithm to compute approximations to the proximal operator of a function.[4]

## See also

* Proximal operator
* Proximal gradient method

## References

1. ↑ Moreau, J. J. (1965). "Proximité et dualité dans un espace hilbertien". Bulletin de la Société Mathématique de France 93: 273–299. doi:10.24033/bsmf.1625. ISSN 0037-9484. https://eudml.org/doc/87067.
2. ↑ 2.0 2.1 Neal Parikh and Stephen Boyd (2013). "Proximal Algorithms". Foundations and Trends in Optimization 1 (3): 123–231. https://web.stanford.edu/~boyd/papers/pdf/prox_algs.pdf. Retrieved 2019-01-29.
3. ↑ Heaton, Howard; Fung, Samy Wu; Osher, Stanley (2022-10-09). "Global Solutions to Nonconvex Problems by Evolution of Hamilton–Jacobi PDEs". arXiv:2202.11014 [math.OC].
4. ↑ Osher, Stanley; Heaton, Howard; Fung, Samy Wu (2023). "A Hamilton–Jacobi-based proximal operator". Proceedings of the National Academy of Sciences 120 (14). doi:10.1073/pnas.2220469120. PMID 36989305. Bibcode: 2023PNAS..12020469O.

## External links

* Hazewinkel, Michiel, ed. (2001), "Moreau envelope function", Encyclopedia of Mathematics, Springer Science+Business Media B.V. / Kluwer Academic Publishers, ISBN 978-1-55608-010-4, https://www.encyclopediaofmath.org/index.php?title=Main_Page
* A Hamilton–Jacobi-based Proximal Operator: a YouTube video explaining an algorithm to approximate the proximal operator

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Moreau envelope. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Moreau_envelope&oldid=3992340"