Functional derivative

Short description: Concept in calculus of variations

In the calculus of variations, a field of mathematical analysis, the functional derivative (or variational derivative)[1] relates a change in a functional (a functional in this sense is a function that acts on functions) to a change in a function on which the functional depends.

In the calculus of variations, functionals are usually expressed in terms of an integral of functions, their arguments, and their derivatives. In an integrand L of a functional, if a function f is varied by adding to it another function δf that is arbitrarily small, and the resulting integrand is expanded in powers of δf, the coefficient of δf in the first order term is called the functional derivative.

For example, consider the functional J[f]=∫abL(x,f(x),f′(x))dx, where f ′(x) ≡ ⁠df/dx⁠. If f is varied by adding to it a function δf, and the resulting integrand L(x, f \+ δf, f ′\+ δf ′) is expanded in powers of δf, then the change in the value of J to first order in δf can be expressed as follows:[1][Note 1] δJ=∫ab(∂L∂fδf(x)+∂L∂f′ddxδf(x))dx=∫ab(∂L∂f−ddx∂L∂f′)δf(x)dx+∂L∂f′(b)δf(b)−∂L∂f′(a)δf(a) where the variation in the derivative, δf ′ was rewritten as the derivative of the variation (δf)′, and integration by parts was used in these derivatives.

## Contents

* 1 Definition
* 1.1 Functional differential
* 1.2 Functional derivative
* 2 Properties
* 3 Determining functional derivatives
* 3.1 Formula
* 3.2 Examples
* 3.2.1 Thomas–Fermi kinetic energy functional
* 3.2.2 Coulomb potential energy functional
* 3.2.3 von Weizsäcker kinetic energy functional
* 3.2.4 Entropy
* 3.2.5 Exponential
* 3.2.6 Functional derivative of a function
* 3.2.7 Functional derivative of iterated function
* 4 Using the delta function as a test function
* 5 Notes
* 6 Footnotes
* 7 References
* 8 External links

## Definition

In this section, the functional differential (or variation or first variation)[Note 2] is defined. Then the functional derivative is defined in terms of the functional differential.

### Functional differential

Suppose B is a Banach space and F is a functional defined on B. The differential of F at a point ρ∈B is the linear functional δF[ρ,⋅] on B defined[2] by the condition that, for all ϕ∈B, F[ρ+ϕ]−F[ρ]=δF[ρ;ϕ]+ε‖ϕ‖ where ε is a real number that depends on ‖ϕ‖ in such a way that ε->0 as ‖ϕ‖->0. This means that δF[ρ,⋅] is the Fréchet derivative of F at ρ.

However, this notion of functional differential is so strong it may not exist,[3] and in those cases a weaker notion, like the Gateaux derivative is preferred. In many practical cases, the functional differential is defined[4] as the directional derivative δF[ρ,ϕ]=limε->0F[ρ+εϕ]−F[ρ]ε=[ddεF[ρ+εϕ]]ε=0. Note that this notion of the functional differential can even be defined without a norm.

In a more general case the function space B appearing as the domain of F is not a vector space, and therefore variations of the form ρ+εϕ do not make sense. In this case we consider a variation α?:(−ε0,ε0)->B of ρ to be a C1-family of functions such that α0=ρ.[Note 3] Denoting the space of all such variations as 𝒱ρ, the functional differential δF[ρ]:𝒱ρ->ℝ is the functional δF[ρ;α]=δF[ρ][α]=limϵ->0F[αϵ]−F[ρ]ϵ=F[α?]′(0)

where F[α?](ϵ)=F[αϵ]. The above then becomes the special case αϵ=ρ+ϵη.[5]

### Functional derivative

In many applications, the domain of the functional F is a space of differentiable functions ρ defined on some space Ω and F is of the form F[ρ]=∫ΩL(x,ρ(x),Dρ(x))dx for some function L(x,ρ(x),Dρ(x)) that may depend on x, the value ρ(x) and the derivative Dρ(x). If this is the case and, moreover, δF[ρ,ϕ] can be written as the integral of ϕ times another function (denoted ⁠δF/δρ⁠) δF[ρ,ϕ]=∫ΩδFδρ(x) ϕ(x) dx then this function ⁠δF/δρ⁠ is called the functional derivative of F at ρ.[6][7] If F is restricted to only certain functions ρ (for example, if there are some boundary conditions imposed) then ϕ is restricted to functions such that ρ+εϕ continues to satisfy these conditions.

Heuristically, ϕ is the change in ρ, so we 'formally' have ϕ=δρ, and then this is similar in form to the total differential of a function F(ρ1,ρ2,…,ρn), dF=∑i=1n∂F∂ρi dρi, where ρ1,ρ2,…,ρn are independent variables. Comparing the last two equations, the functional derivative δFδρ(x) has a role similar to that of the partial derivative ∂F∂ρi, where the variable of integration x is like a continuous version of the summation index i.[8] One thinks of ⁠δF/δρ⁠ as the gradient of F at the point ρ, so the value ⁠δF/δρ(x)⁠ measures how much the functional F will change if the function ρ is changed at the point x. Hence the formula ∫δFδρ(x)ϕ(x)dx is regarded as the directional derivative at point ρ in the direction of ϕ. This is analogous to vector calculus, where the inner product of a vector v with the gradient gives the directional derivative in the direction of v.

## Properties

Like the derivative of a function, the functional derivative satisfies the following properties, where F[ρ] and G[ρ] are functionals:[Note 4]

* Linearity:[9] δ(λF+μG)[ρ]δρ(x)=λδF[ρ]δρ(x)+μδG[ρ]δρ(x), where λ, μ are constants.
* Product rule:[10] δ(FG)[ρ]δρ(x)=δF[ρ]δρ(x)G[ρ]+F[ρ]δG[ρ]δρ(x),
* Chain rules:
* If F is a functional and G another functional, then[11] δF[G[ρ]]δρ(y)=∫dxδF[G]δG(x)G=G[ρ]⋅δG[ρ](x)δρ(y) .
* If G is an ordinary differentiable function (local functional) g, then this reduces to[12] δF[g(ρ)]δρ(y)=δF[g(ρ)]δg[ρ(y)] dg(ρ)dρ(y) .

## Determining functional derivatives

A formula to determine functional derivatives for a common class of functionals can be written as the integral of a function and its derivatives. This is a generalization of the Euler–Lagrange equation: indeed, the functional derivative was introduced in physics within the derivation of the Lagrange equation of the second kind from the principle of least action in Lagrangian mechanics (18th century). The first three examples below are taken from density functional theory (20th century), the fourth from statistical mechanics (19th century).

### Formula

Given a functional F[ρ]=∫f(r,ρ(r),∇ρ(r))dr, and a function ϕ(r) that vanishes on the boundary of the region of integration, from a previous section Definition, ∫δFδρ(r)ϕ(r)dr=[ddε∫f(r,ρ+εϕ,∇ρ+ε∇ϕ)dr]ε=0=∫(∂f∂ρϕ+∂f∂∇ρ⋅∇ϕ)dr=∫[∂f∂ρϕ+∇⋅(∂f∂∇ρϕ)−(∇⋅∂f∂∇ρ)ϕ]dr=∫[∂f∂ρϕ−(∇⋅∂f∂∇ρ)ϕ]dr=∫(∂f∂ρ−∇⋅∂f∂∇ρ)ϕ(r) dr.

The second line is obtained using the total derivative, where ⁠∂f/∂∇ρ⁠ is a derivative of a scalar with respect to a vector.[Note 5]

The third line was obtained by use of a product rule for divergence. The fourth line was obtained using the divergence theorem and the condition that ϕ=0 on the boundary of the region of integration. Since ϕ is also an arbitrary function, applying the fundamental lemma of calculus of variations to the last line, the functional derivative is δFδρ(r)=∂f∂ρ−∇⋅∂f∂∇ρ

where ρ = ρ(r) and f = f(r, ρ, ∇ρ). This formula is for the case of the functional form given by F[ρ] at the beginning of this section. For other functional forms, the definition of the functional derivative can be used as the starting point for its determination. (See the example Coulomb potential energy functional.)

The above equation for the functional derivative can be generalized to the case that includes higher dimensions and higher order derivatives. The functional would be, F[ρ(r)]=∫f(r,ρ(r),∇ρ(r),∇(2)ρ(r),…,∇(N)ρ(r))dr,

where the vector r ∈ Rn, and ∇(i) is a tensor whose ni components are partial derivative operators of order i, [∇(i)]α1α2⋯αi=∂i∂rα1∂rα2⋯∂rαiwhereα1,α2,…,αi=1,2,…,n .[Note 6]

An analogous application of the definition of the functional derivative yields δF[ρ]δρ=∂f∂ρ−∇⋅∂f∂(∇ρ)+∇(2)⋅∂f∂(∇(2)ρ)+…+(−1)N∇(N)⋅∂f∂(∇(N)ρ)=∂f∂ρ+∑i=1N(−1)i∇(i)⋅∂f∂(∇(i)ρ) .

In the last two equations, the ni components of the tensor ∂f∂(∇(i)ρ) are partial derivatives of f with respect to partial derivatives of ρ, [∂f∂(∇(i)ρ)]α1α2⋯αi=∂f∂ρα1α2⋯αi where ρα1α2⋯αi≡∂iρ∂rα1∂rα2⋯∂rαi, and the tensor scalar product is, ∇(i)⋅∂f∂(∇(i)ρ)=∑α1,α2,⋯,αi=1n ∂i∂rα1∂rα2⋯∂rαi ∂f∂ρα1α2⋯αi . [Note 7]

### Examples

#### Thomas–Fermi kinetic energy functional

The Thomas–Fermi model of 1927 used a kinetic energy functional for a noninteracting uniform electron gas in a first attempt of density-functional theory of electronic structure: TTF[ρ]=CF∫ρ53(𝐫)d𝐫. Since the integrand of TTF[ρ] does not involve derivatives of ρ(r), the functional derivative of TTF[ρ] is,[13] δTTFδρ(r)=CF∂ρ53(𝐫)∂ρ(𝐫)=53CFρ23(𝐫).

#### Coulomb potential energy functional

The electron–nucleus potential energy is V[ρ]=∫ρ(r)|r| dr.

Applying the definition of functional derivative, ∫δVδρ(r) ϕ(r) dr=[ddε∫ρ(r)+εϕ(r)|r| dr]ε=0=∫ϕ(r)|r| dr. So, δVδρ(r)=1|r| .

The functional derivative of the classical part of the electron–electron interaction (often called Hartree energy) is J[ρ]=12∬ρ(𝐫)ρ(𝐫′)|𝐫−𝐫′|d𝐫d𝐫′. From the definition of the functional derivative, ∫δJδρ(r)ϕ(r)dr=[d dεJ[ρ+εϕ]]ε=0=[d dε(12∬[ρ(r)+εϕ(r)][ρ(r′)+εϕ(r′)]|r−r′|drdr′)]ε=0=12∬ρ(r′)ϕ(r)|r−r′|drdr′+12∬ρ(r)ϕ(r′)|r−r′|drdr′ The first and second terms on the right hand side of the last equation are equal, since r and r′ in the second term can be interchanged without changing the value of the integral. Therefore, ∫δJδρ(r)ϕ(r)dr=∫(∫ρ(r′)|r−r′|dr′)ϕ(r)dr and the functional derivative of the electron-electron Coulomb potential energy functional J[ρ] is,[14] δJδρ(r)=∫ρ(r′)|r−r′|dr′.

The second functional derivative is δ2J[ρ]δρ(𝐫′)δρ(𝐫)=∂∂ρ(𝐫′)(ρ(𝐫′)|𝐫−𝐫′|)=1|𝐫−𝐫′|.

#### von Weizsäcker kinetic energy functional

In 1935 von Weizsäcker proposed to add a gradient correction to the Thomas-Fermi kinetic energy functional to make it better suit a molecular electron cloud: TW[ρ]=18∫∇ρ(𝐫)⋅∇ρ(𝐫)ρ(𝐫)d𝐫=∫tW(𝐫) d𝐫, where tW≡18∇ρ⋅∇ρρand ρ=ρ(r) . Using a previously derived formula for the functional derivative, δTWδρ=∂tW∂ρ−∇⋅∂tW∂∇ρ=−18∇ρ⋅∇ρρ2−(14∇2ρρ−14∇ρ⋅∇ρρ2)where ∇2=∇⋅∇ , and the result is,[15] δTWδρ= 18∇ρ⋅∇ρρ2−14∇2ρρ .

#### Entropy

The entropy of a discrete random variable is a functional of the probability mass function.

H[p(x)]=−∑xp(x)log⁡p(x) Thus, ∑xδHδp(x)ϕ(x)=[ddεH[p(x)+εϕ(x)]]ε=0=[−ddε∑x[p(x)+εϕ(x)] log⁡[p(x)+εϕ(x)]]ε=0=−∑x[1+log⁡p(x)] ϕ(x). Thus, δHδp(x)=−1−log⁡p(x).

#### Exponential

Let F[φ(x)]=e∫φ(x)g(x)dx.

Using the delta function as a test function, δF[φ(x)]δφ(y)=limε->0F[φ(x)+εδ(x−y)]−F[φ(x)]ε=limε->0e∫(φ(x)+εδ(x−y))g(x)dx−e∫φ(x)g(x)dxε=e∫φ(x)g(x)dxlimε->0eε∫δ(x−y)g(x)dx−1ε=e∫φ(x)g(x)dxlimε->0eεg(y)−1ε=e∫φ(x)g(x)dxg(y).

Thus, δF[φ(x)]δφ(y)=g(y)F[φ(x)].

This is particularly useful in calculating the correlation functions from the partition function in quantum field theory.

#### Functional derivative of a function

A function can be written in the form of an integral like a functional. For example, ρ(r)=F[ρ]=∫ρ(r′)δ(r−r′)dr′. Since the integrand does not depend on derivatives of ρ, the functional derivative of ρ(r) is, δρ(r)δρ(r′)≡δFδρ(r′)=∂ ∂ρ(r′)[ρ(r′)δ(r−r′)]=δ(r−r′).

#### Functional derivative of iterated function

The functional derivative of the iterated function f(f(x)) is given by: δf(f(x))δf(y)=f′(f(x))δ(x−y)+δ(f(x)−y) and δf(f(f(x)))δf(y)=f′(f(f(x))(f′(f(x))δ(x−y)+δ(f(x)−y))+δ(f(f(x))−y)

In general: δfN(x)δf(y)=f′(fN−1(x))δfN−1(x)δf(y)+δ(fN−1(x)−y)

Putting in N = 0 gives: δf−1(x)δf(y)=−δ(f−1(x)−y)f′(f−1(x))

## Using the delta function as a test function

In physics, it is common to use the Dirac delta function δ(x−y) in place of a generic test function ϕ(x), for yielding the functional derivative at the point y (this is a point of the whole functional derivative as a partial derivative is a component of the gradient):[16] δF[ρ(x)]δρ(y)=limε->0F[ρ(x)+εδ(x−y)]−F[ρ(x)]ε.

This works in cases when F[ρ(x)+εf(x)] formally can be expanded as a series (or at least up to first order) in ε. The formula is however not mathematically rigorous, since F[ρ(x)+εδ(x−y)] is usually not even defined.

The definition given in a previous section is based on a relationship that holds for all test functions ϕ(x), so one might think that it should hold also when ϕ(x) is chosen to be a specific function such as the delta function. However, the latter is not a valid test function (it is not even a proper function).

In the definition, the functional derivative describes how the functional F[ρ(x)] changes as a result of a small change in the entire function ρ(x). The particular form of the change in ρ(x) is not specified, but it should stretch over the whole interval on which x is defined. Employing the particular form of the perturbation given by the delta function has the meaning that ρ(x) is varied only in the point y. Except for this point, there is no variation in ρ(x).

## Notes

1. ↑ According to Giaquinta & Hildebrandt (1996), p. 18, this notation is customary in physical literature.
2. ↑ Called first variation in (Giaquinta Hildebrandt), variation or first variation in (Courant Hilbert), variation or differential in (Gelfand Fomin) and differential in (Parr Yang).
3. ↑ cf. homotopy.
4. ↑ Here the notation δFδρ(x)≡δFδρ(x) is introduced.
5. ↑ For a three-dimensional Cartesian coordinate system, ∂f∂∇ρ=∂f∂ρxi^+∂f∂ρyj^+∂f∂ρzk^, where ρx=∂ρ∂x, ρy=∂ρ∂y, ρz=∂ρ∂z and i^, j^, k^ are unit vectors along the x, y, z axes.
6. ↑ For example, for the case of three dimensions (n = 3) and second order derivatives (i = 2), the tensor ∇(2) has components, [∇(2)]αβ=∂2∂rα∂rβwhere α and β can be 1,2,3.
7. ↑ For example, for the case n = 3 and i = 2, the tensor scalar product is, ∇(2)⋅∂f∂(∇(2)ρ)=∑α,β=13 ∂2∂rα∂rβ∂f∂ραβ,where ραβ≡∂2ρ∂rα∂rβ.

## Footnotes

1. ↑ 1.0 1.1 Giaquinta & Hildebrandt (1996), p. 18
2. ↑ Gelfand & Fomin (2000), p. 11.
3. ↑ Giaquinta & Hildebrandt (1996), p. 10.
4. ↑ Giaquinta & Hildebrandt (1996), p. 10.
5. ↑ Terek, Ivo (2019-06-12). "Introductory Variational Calculus on Manifolds" (PDF). https://web.williams.edu/Mathematics/it3/texts/var_noether.pdf.
6. ↑ Parr & Yang (1989), p. 246, Eq. A.2.
7. ↑ Greiner & Reinhardt (1996), p. 36,37.
8. ↑ Parr & Yang (1989), p. 246.
9. ↑ Parr & Yang (1989), p. 247, Eq. A.3.
10. ↑ Parr & Yang (1989), p. 247, Eq. A.4.
11. ↑ Greiner & Reinhardt (1996), p. 38, Eq. 6.
12. ↑ Greiner & Reinhardt (1996), p. 38, Eq. 7.
13. ↑ Parr & Yang (1989), p. 247, Eq. A.6.
14. ↑ Parr & Yang (1989), p. 248, Eq. A.11.
15. ↑ Parr & Yang (1989), p. 247, Eq. A.9.
16. ↑ Greiner & Reinhardt (1996), p. 37

## References

* Courant, Richard; Hilbert, David (1953). "Chapter IV. The Calculus of Variations". Methods of Mathematical Physics. I (First English ed.). New York, New York: Interscience Publishers, Inc. pp. 164–274. ISBN 978-0471504474. .
* Frigyik, Béla A.; Srivastava, Santosh; Gupta, Maya R. (January 2008), Introduction to Functional Derivatives, UWEE Tech Report, UWEETR-2008-0001, Seattle, WA: Department of Electrical Engineering at the University of Washington, pp. 7, https://www.ee.washington.edu/techsite/papers/documents/UWEETR-2008-0001.pdf, retrieved 2013-10-23 .
* Gelfand, I. M.; Fomin, S. V. (2000), Calculus of variations, translated and edited by Richard A. Silverman (Revised English ed.), Mineola, N.Y.: Dover Publications, ISBN 978-0486414485, http://store.doverpublications.com/0486414485.html .
* Giaquinta, Mariano; Hildebrandt, Stefan (1996), Calculus of Variations 1. The Lagrangian Formalism, Grundlehren der Mathematischen Wissenschaften, 310 (1st ed.), Berlin: Springer-Verlag, ISBN 3-540-50625-X .
* Greiner, Walter; Reinhardt, Joachim (1996), "Section 2.3 – Functional derivatives", Field quantization, With a foreword by D. A. Bromley, Berlin–Heidelberg–New York: Springer-Verlag, pp. 36–38, ISBN 3-540-59179-6, https://archive.org/details/fieldquantizatio0000grei/page/36 .
* Parr, R. G.; Yang, W. (1989). "Appendix A, Functionals". Density-Functional Theory of Atoms and Molecules. New York: Oxford University Press. pp. 246–254. ISBN 978-0195042795. https://books.google.com/books?id=mGOpScSIwU4C&q=Density-Functional+Theory+of+Atoms+and+Molecules.

## External links

* Hazewinkel, Michiel, ed. (2001), "Functional derivative", Encyclopedia of Mathematics, Springer Science+Business Media B.V. / Kluwer Academic Publishers, ISBN 978-1-55608-010-4, https://www.encyclopediaofmath.org/index.php?title=p/f042040

* v
* t
* e

Functional analysis (topics)

Topological vector spaces|

* Asplund
* Banach (list)
* Banach lattice
* Barrelled
* Bornological
* Brauner
* F-space
* Fréchet (tame)
* Hilbert (Inner product space
* Polarization identity)
* LF-space
* Locally convex (Seminorms/Minkowski functionals)
* Mackey
* Montel
* Nuclear
* Normed (norm)
* Quasinormed
* Reflexive
* Riesz
* Smith
* Stereotype
* Strictly convex
* Webbed
* Topological tensor product (of Hilbert spaces)

Topologies of function spaces|

* Dual
* Dual space (Dual norm)
* Operator
* Ultraweak
* Weak (polar
* operator)
* Mackey
* Strong (polar
* operator)
* Ultrastrong
* Uniform convergence

Linear operators|

* Adjoint
* Bilinear (form
* operator
* sesquilinear)
* (Un)Bounded
* Closed
* Compact (on Hilbert spaces)
* (Dis)Continuous
* Densely defined
* Fredholm
* Hilbert–Schmidt
* Functionals (positive)
* Normal
* Nuclear
* Self-adjoint
* Strictly singular
* Trace class
* Transpose
* Unitary

Operator theory|

* Banach algebras
* C*-algebras
* Spectrum (C*-algebra
* radius)
* Spectral theory (of ODEs
* Spectral theorem)
* Polar decomposition
* Singular value decomposition

Theorems|

* Banach–Alaoglu
* Banach–Mazur
* Banach–Saks
* Banach–Schauder (open mapping)
* Banach–Steinhaus (Uniform boundedness)
* Bessel's inequality
* Cauchy–Schwarz inequality
* Closed graph
* Closed range
* Eberlein–Šmulian
* Freudenthal spectral
* Gelfand–Mazur
* Gelfand–Naimark
* Goldstine
* Hahn–Banach (hyperplane separation)
* Kakutani fixed-point
* Krein–Milman
* Lomonosov's invariant subspace
* Mackey–Arens
* Mazur's lemma
* M. Riesz extension
* Riesz representation
* Parseval's identity
* Schauder fixed-point

Analysis|

* Abstract Wiener space
* Bochner space
* Differentiation in Fréchet spaces
* Derivatives (Fréchet
* Gateaux
* functional
* holomorphic)
* Integrals (Bochner
* Dunford
* Gelfand–Pettis
* regulated
* Paley–Wiener
* weak)
* Functional calculus (Borel
* continuous
* holomorphic)
* Inverse function theorem (Nash–Moser theorem)
* Measures (Lebesgue
* Projection-valued
* Vector)
* Weakly measurable function

Types of sets|

* Absolutely convex
* Absorbing
* Balanced
* Bounded
* Convex
* Convex cone (subset)
* Linear cone (subset)
* Radial
* Star-shaped
* Symmetric
* Zonotope

Subsets / set operations|

* Algebraic interior (core)
* Bounding points
* Convex hull
* Extreme point
* Interior
* Minkowski addition
* Polar

Template:Analysis in topological vector spaces

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Functional derivative. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Functional_derivative&oldid=4226224"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template