Convex conjugate

Inmathematicsandmathematical optimization,theconvex conjugateof a function is a generalization of theLegendre transformationwhich applies to non-convex functions. It is also known asLegendre–Fenchel transformation,Fenchel transformation,orFenchel conjugate(afterAdrien-Marie LegendreandWerner Fenchel). The convex conjugate is widely used for constructing thedual probleminoptimization theory,thus generalizingLagrangian duality.

Definition

Let$X$be arealtopological vector spaceand let$X^{*}$be thedual spaceto$X$.Denote by

$\langle \cdot,\cdot \rangle:X^{*}\times X\to \mathbb {R}$

the canonicaldual pairing,which is defined by$\left\langle x^{*},x\right\rangle \mapsto x^{*}(x).$

For a function$f:X\to \mathbb {R} \cup \{-\infty,+\infty \}$taking values on theextended real number line,itsconvex conjugateis the function

$f^{*}:X^{*}\to \mathbb {R} \cup \{-\infty,+\infty \}$

whose value at$x^{*}\in X^{*}$is defined to be thesupremum:

$f^{*}\left(x^{*}\right):=\sup \left\{\left\langle x^{*},x\right\rangle -f(x)~\colon ~x\in X\right\},$

or, equivalently, in terms of theinfimum:

$f^{*}\left(x^{*}\right):=-\inf \left\{f(x)-\left\langle x^{*},x\right\rangle ~\colon ~x\in X\right\}.$

This definition can be interpreted as an encoding of theconvex hullof the function'sepigraphin terms of itssupporting hyperplanes.

Examples

For more examples, see§ Table of selected convex conjugates.

• The convex conjugate of anaffine function$f(x)=\left\langle a,x\right\rangle -b$is$$f^{*}\left(x^{*}\right)={\begin{cases}b,&x^{*}=a\\+\infty,&x^{*}\neq a.\end{cases}}$$
• The convex conjugate of apower function$f(x)={\frac {1}{p}}|x|^{p},1<p<\infty$is$$f^{*}\left(x^{*}\right)={\frac {1}{q}}|x^{*}|^{q},1<q<\infty,{\text{where}}{\tfrac {1}{p}}+{\tfrac {1}{q}}=1.$$
• The convex conjugate of theabsolute valuefunction$f(x)=\left|x\right|$is$$f^{*}\left(x^{*}\right)={\begin{cases}0,&\left|x^{*}\right|\leq 1\\\infty,&\left|x^{*}\right|>1.\end{cases}}$$
• The convex conjugate of theexponential function$f(x)=e^{x}$is$$f^{*}\left(x^{*}\right)={\begin{cases}x^{*}\ln x^{*}-x^{*},&x^{*}>0\\0,&x^{*}=0\\\infty,&x^{*}<0.\end{cases}}$$

The convex conjugate and Legendre transform of the exponential function agree except that thedomainof the convex conjugate is strictly larger as the Legendre transform is only defined for positive real numbers.

Connection with expected shortfall (average value at risk)

Seethis article for example.

LetFdenote acumulative distribution functionof arandom variableX.Then (integrating by parts), $$f(x):=\int _{-\infty }^{x}F(u)\,du=\operatorname {E} \left[\max(0,x-X)\right]=x-\operatorname {E} \left[\min(x,X)\right]$$ has the convex conjugate $$f^{*}(p)=\int _{0}^{p}F^{-1}(q)\,dq=(p-1)F^{-1}(p)+\operatorname {E} \left[\min(F^{-1}(p),X)\right]=pF^{-1}(p)-\operatorname {E} \left[\max(0,F^{-1}(p)-X)\right].$$

Ordering

A particular interpretation has the transform $$f^{\text{inc}}(x):=\arg \sup _{t}t\cdot x-\int _{0}^{1}\max\{t-f(u),0\}\,du,$$ as this is a nondecreasing rearrangement of the initial functionf;in particular,$f^{\text{inc}}=f$forfnondecreasing.

Properties

The convex conjugate of aclosed convex functionis again a closed convex function. The convex conjugate of apolyhedral convex function(a convex function withpolyhedralepigraph) is again a polyhedral convex function.

Order reversing

Declare that$f\leq g$if and only if$f(x)\leq g(x)$for all$x.$Then convex-conjugation isorder-reversing,which by definition means that if$f\leq g$then$f^{*}\geq g^{*}.$

For a family of functions$\left(f_{\ Alpha }\right)_{\ Alpha }$it follows from the fact that supremums may be interchanged that

$\left(\inf _{\ Alpha }f_{\ Alpha }\right)^{*}(x^{*})=\sup _{\ Alpha }f_{\ Alpha }^{*}(x^{*}),$

and from themax–min inequalitythat

$\left(\sup _{\ Alpha }f_{\ Alpha }\right)^{*}(x^{*})\leq \inf _{\ Alpha }f_{\ Alpha }^{*}(x^{*}).$

Biconjugate

The convex conjugate of a function is alwayslower semi-continuous.Thebiconjugate$f^{**}$(the convex conjugate of the convex conjugate) is also theclosed convex hull,i.e. the largestlower semi-continuousconvex function with$f^{**}\leq f.$ Forproper functions$f,$

$f=f^{**}$if and only if$f$is convex and lower semi-continuous, by theFenchel–Moreau theorem.

Fenchel's inequality

For any functionfand its convex conjugatef*,Fenchel's inequality(also known as theFenchel–Young inequality) holds for every$x\in X$and$p\in X^{*}$:

$\left\langle p,x\right\rangle \leq f(x)+f^{*}(p).$

Furthermore, the equality holds only when$p\in \partial f(x)$. The proof follows from the definition of convex conjugate:$f^{*}(p)=\sup _{\tilde {x}}\left\{\langle p,{\tilde {x}}\rangle -f({\tilde {x}})\right\}\geq \langle p,x\rangle -f(x).$

Convexity

For two functions$f_{0}$and$f_{1}$and a number$0\leq \lambda \leq 1$the convexity relation

$\left((1-\lambda )f_{0}+\lambda f_{1}\right)^{*}\leq (1-\lambda )f_{0}^{*}+\lambda f_{1}^{*}$

holds. The${*}$operation is a convex mapping itself.

Infimal convolution

Theinfimal convolution(or epi-sum) of two functions$f$and$g$is defined as

$\left(f\operatorname {\Box } g\right)(x)=\inf \left\{f(x-y)+g(y)\mid y\in \mathbb {R} ^{n}\right\}.$

Let$f_{1},\ldots,f_{m}$beproper,convex andlower semicontinuousfunctions on$\mathbb {R} ^{n}.$Then the infimal convolution is convex and lower semicontinuous (but not necessarily proper),and satisfies

$\left(f_{1}\operatorname {\Box } \cdots \operatorname {\Box } f_{m}\right)^{*}=f_{1}^{*}+\cdots +f_{m}^{*}.$

The infimal convolution of two functions has a geometric interpretation: The (strict)epigraphof the infimal convolution of two functions is theMinkowski sumof the (strict) epigraphs of those functions.

Maximizing argument

If the function$f$is differentiable, then its derivative is the maximizing argument in the computation of the convex conjugate:

$f^{\prime }(x)=x^{*}(x):=\arg \sup _{x^{*}}{\langle x,x^{*}\rangle }-f^{*}\left(x^{*}\right)$and

$f^{{*}\prime }\left(x^{*}\right)=x\left(x^{*}\right):=\arg \sup _{x}{\langle x,x^{*}\rangle }-f(x);$

hence

$x=\nabla f^{*}\left(\nabla f(x)\right),$

$x^{*}=\nabla f\left(\nabla f^{*}\left(x^{*}\right)\right),$

and moreover

$f^{\prime \prime }(x)\cdot f^{{*}\prime \prime }\left(x^{*}(x)\right)=1,$

$f^{{*}\prime \prime }\left(x^{*}\right)\cdot f^{\prime \prime }\left(x(x^{*})\right)=1.$

Scaling properties

If for some$\gamma >0,$$g(x)=\ Alpha +\beta x+\gamma \cdot f\left(\lambda x+\delta \right)$,then

$g^{*}\left(x^{*}\right)=-\ Alpha -\delta {\frac {x^{*}-\beta }{\lambda }}+\gamma \cdot f^{*}\left({\frac {x^{*}-\beta }{\lambda \gamma }}\right).$

Behavior under linear transformations

Let$A:X\to Y$be abounded linear operator.For any convex function$f$on$X,$

$\left(Af\right)^{*}=f^{*}A^{*}$

where

$(Af)(y)=\inf\{f(x):x\in X,Ax=y\}$

is the preimage of$f$with respect to$A$and$A^{*}$is theadjoint operatorof$A.$

A closed convex function$f$is symmetric with respect to a given set$G$oforthogonal linear transformations,

$f(Ax)=f(x)$for all$x$and all$A\in G$

if and only if its convex conjugate$f^{*}$is symmetric with respect to$G.$

Table of selected convex conjugates

The following table provides Legendre transforms for many common functions as well as a few useful properties.

$g(x)$	$\operatorname {dom} (g)$	$g^{*}(x^{*})$	$\operatorname {dom} (g^{*})$
$f(ax)$(where$a\neq 0$)	$X$	$f^{*}\left({\frac {x^{*}}{a}}\right)$	$X^{*}$
$f(x+b)$	$X$	$f^{*}(x^{*})-\langle b,x^{*}\rangle$	$X^{*}$
$af(x)$(where$a>0$)	$X$	$af^{*}\left({\frac {x^{*}}{a}}\right)$	$X^{*}$
$\ Alpha +\beta x+\gamma \cdot f(\lambda x+\delta )$	$X$	$-\ Alpha -\delta {\frac {x^{*}-\beta }{\lambda }}+\gamma \cdot f^{*}\left({\frac {x^{*}-\beta }{\gamma \lambda }}\right)\quad (\gamma >0)$	$X^{*}$
${\frac {|x|^{p}}{p}}$(where$p>1$)	$\mathbb {R}$	${\frac {|x^{*}|^{q}}{q}}$(where${\frac {1}{p}}+{\frac {1}{q}}=1$)	$\mathbb {R}$
${\frac {-x^{p}}{p}}$(where$0<p<1$)	$\mathbb {R} _{+}$	${\frac {-(-x^{*})^{q}}{q}}$(where${\frac {1}{p}}+{\frac {1}{q}}=1$)	$\mathbb {R} _{--}$
${\sqrt {1+x^{2}}}$	$\mathbb {R}$	$-{\sqrt {1-(x^{*})^{2}}}$	$[-1,1]$
$-\log(x)$	$\mathbb {R} _{++}$	$-(1+\log(-x^{*}))$	$\mathbb {R} _{--}$
$e^{x}$	$\mathbb {R}$	${\begin{cases}x^{*}\log(x^{*})-x^{*}&{\text{if }}x^{*}>0\\0&{\text{if }}x^{*}=0\end{cases}}$	$\mathbb {R} _{+}$
$\log \left(1+e^{x}\right)$	$\mathbb {R}$	${\begin{cases}x^{*}\log(x^{*})+(1-x^{*})\log(1-x^{*})&{\text{if }}0<x^{*}<1\\0&{\text{if }}x^{*}=0,1\end{cases}}$	$[0,1]$
$-\log \left(1-e^{x}\right)$	$\mathbb {R} _{--}$	${\begin{cases}x^{*}\log(x^{*})-(1+x^{*})\log(1+x^{*})&{\text{if }}x^{*}>0\\0&{\text{if }}x^{*}=0\end{cases}}$	$\mathbb {R} _{+}$

See also

• Dual problem
• Fenchel's duality theorem
• Legendre transformation
• Young's inequality for products

References

• Arnol'd, Vladimir Igorevich(1989).Mathematical Methods of Classical Mechanics(Second ed.). Springer.ISBN0-387-96890-3.MR0997295.
• Rockafellar, R. Tyrrell;Wets, Roger J.-B.(26 June 2009).Variational Analysis.Grundlehren der mathematischen Wissenschaften. Vol. 317. Berlin New York:Springer Science & Business Media.ISBN9783642024313.OCLC883392544.
• Rockafellar, R. Tyrell(1970).Convex Analysis.Princeton: Princeton University Press.ISBN0-691-01586-4.MR0274683.

Further reading

• Touchette, Hugo (2014-10-16)."Legendre-Fenchel transforms in a nutshell"(PDF).Archived fromthe original(PDF)on 2017-04-07.Retrieved2017-01-09.

• Touchette, Hugo (2006-11-21)."Elements of convex analysis"(PDF).Archived fromthe original(PDF)on 2015-05-26.Retrieved2008-03-26.

• "Legendre and Legendre-Fenchel transforms in a step-by-step explanation".Retrieved2013-05-18.

• Ellerman, David Patterson(1995-03-21)."Chapter 12: Parallel Addition, Series-Parallel Duality, and Financial Mathematics".Intellectual Trespassing as a Way of Life: Essays in Philosophy, Economics, and Mathematics(PDF).The worldly philosophy: studies in intersection of philosophy and economics.Rowman & Littlefield Publishers, Inc.pp. 237–268.ISBN0-8476-7932-2.Archived(PDF)from the original on 2016-03-05.Retrieved2019-08-09.Series G - Reference, Information and Interdisciplinary Subjects Series[1](271 pages)

• Ellerman, David Patterson(May 2004) [1995-03-21]."Introduction to Series-Parallel Duality"(PDF).University of California at Riverside.CiteSeerX10.1.1.90.3666.Archivedfrom the original on 2019-08-10.Retrieved2019-08-09.[2](24 pages)