Convexity

A set is convex if, for all and all :

A function is convex if its graph lies above every chord joining two points:

Convexity is strict if the inequality is strict whenever and .

Intuition

Geometrically, a convex set has no holes or indentations: the segment between any two points stays inside. A convex function curves upward: every chord between two graph points dominates the function.

This property is central in optimization: on a convex domain, any local minimum of a convex function is global. There are no local traps.

Characterizations

If is convex and is a random variable with well-defined quantities:

This is the standard tool linking averages and nonlinear transformations — useful in probability, finance, and risk analysis.

is convex if and only if, for all :

The graph dominates its tangents: the linearization always underestimates the function.

If is , it is convex on a convex open set if and only if:

for every in the domain. This is the most practical test for verifying convexity of a loss or objective function.

Properties

Property	Statement	Consequence
Minima	Every local minimum is global	Optimization is well posed without local traps
Combinations	Nonnegative weighted sum of convex functions	Multi-objective goals remain convex
Composition	convex if is convex increasing and is convex	Practical rule for checking compositions
Pointwise maximum	convex if each is	Worst-case constraints preserve convexity
Epigraph	convex ⟺ its epigraph is a convex set	Geometric link between functions and sets

Examples

Function / set	Convexity	Comment
	Convex	Constant Hessian
	Convex	Not differentiable at 0, but convex in the broad sense
norms	Convex	Unit balls are convex; basis for regularization constraints
	Convex	Strictly positive second derivative
on	Convex	Appears in log-likelihoods and entropy
	Concave	Negative of a convex function
MSE	Convex in parameters	Foundation of linear regression
Hinge loss	Convex	Maximum-margin SVM
Half-space	Convex set	Building block of linear constraints

Applications

Convexity is the foundation of convex optimization: problems where the objective and feasible set are convex. Classical algorithms (gradient, interior-point methods) then converge to the global optimum under moderate assumptions.

Direct links on this site:

Linear regression — least squares minimizes a strictly convex quadratic function (when is positive definite).
Convex optimization — gradient descent, constraints, Lagrange multipliers; convexity ensures these methods target the right optimum.
Portfolio allocation — Markowitz formulates a convex quadratic program under linear constraints.
Machine learning — MSE, hinge loss, and cross-entropy (via the log) rely on convex structure in the parameters; this is what makes training linear models and SVMs reliable.

Finance link (distinct): bond convexity measures the curvature of a bond price with respect to yields (second derivative of price). It uses the language of convexity but is not the same topic as mathematical convexity — see asset classes.

Limits and pitfalls

Situation	Issue	Consequence
Non-convex function	E.g. sigmoid, deep network losses	Local minima; convergence depends on initialization
Quasi-convexity	Convex level sets, but function not necessarily convex	Easier than general optimization, but without Jensen's tools
Non-convex constraints	E.g. cardinality constraint	The global problem becomes NP-hard in general
Convexity in parameters only	Loss may be convex in while the model stays linear	Do not confuse loss convexity with predictor expressiveness

Notion	Idea
Concavity	concave ⟺ convex; maximize rather than minimize
Strongly convex function	Hessian bounded below by with ; faster convergence
Convex set vs convex problem	A convex domain is not enough: the objective must also be convex
Convex optimization	Algorithmic study of convex problems — see Convex optimization