Infinitesimal calculus MOC

Lagrange multiplier

Lagrange multipliers are an optimisation technique under a constraint particularly useful when it is impossible or difficult to reduce the function to be optimised to a single variable function. It forms the basis of the Lagrangian function.¹

Statement

Given a function to be optimised 𝑓(⃗𝐯) and constraining function 𝑔(⃗𝐯) =𝑘, a maximising or minimising input ⃗𝐜 will satisfy

∇𝑓(⃗𝐜)=𝜆∇𝑔(⃗𝐜)

where 𝜆 ≠0 is called the Lagrangian multiplier.

Multiple constraints

In the case of optimising 𝑓(⃗𝐯) under multiple constraints

𝑔1(⃗𝐯)=𝑘1,𝑔2(⃗𝐯)=𝑘2,…,𝑔𝑛(⃗𝐯)=𝑘𝑛

the equation to be satisfied becomes

∇𝑓(⃗𝐜)=𝑛∑𝑖=1𝜆𝑖∇𝑔𝑛(𝑐)=𝜆1∇𝑔1(⃗𝐜)+𝜆2∇𝑔2(⃗𝐜)+⋯+𝜆𝑛∇𝑔𝑛(⃗𝐜)

Intuitive justification

The constraint 𝑔(⃗𝐯) =𝑘 forms a Level set of 𝑔(⃗𝐯). Therefore any inputs to 𝑓 which satisfy the constraint correspond to intersections of the level curve 𝑔(⃗𝐯) =𝑘 and some level curve 𝑓(⃗𝐯) =𝑙. For any optimising (i.e. maximising or minimising) input of ⃗𝐜, the two level curves will be tangent. Since Gradient vectors are perpendicular to level curves, this necessarily implies the the gradients ∇𝑓(⃗𝐜) and ∇𝑔(⃗𝐜) are parallel, and hence there exists some nonzero 𝜆 such that²

∇𝑓(⃗𝐜)=𝜆∇𝑔(⃗𝐜)

Usage

In order to solve an optimisation problem using Lagrangian multipliers with input of dimension 𝑛 (i.e. 𝑓 :ℝ𝑛 →ℝ), one must solve a system of 𝑛 +1 equations.

⎡⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢⎣𝜕𝑓𝜕𝑥1(𝑥1,𝑥2…𝑥𝑛)𝜕𝑓𝜕𝑥2(𝑥1,𝑥2…𝑥𝑛)⋮𝜕𝑓𝜕𝑥𝑛(𝑥1,𝑥2…𝑥𝑛)𝑔(𝑥1,𝑥2…𝑥𝑛)⎤⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥⎦=⎡⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢⎣𝜆𝜕𝑔𝜕𝑥1(𝑥1,𝑥2…𝑥𝑛)𝜆𝜕𝑔𝜕𝑥2(𝑥1,𝑥2…𝑥𝑛)⋮𝜆𝜕𝑔𝜕𝑥𝑛(𝑥1,𝑥2…𝑥𝑛)𝑘⎤⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥⎦

Significance of the multiplier

The lambda multiplier 𝜆 is not an arbitrary, meaningless value. It is the derivative of the optimised value with respect to the constraining value 𝑘 where 𝑔(⃗𝐯) =𝑘 is the constraint.

See also

• Duality of optimisation problems

Practice problems

• 2022. MATH1011: Multivariable calculus, pp. 65–66 (example 4.29; exercise 4.4.1)
• 2016. Calculus, pp. 1017–1018 (§14.8 exercises)
• 2023. Advanced Mathematical Methods, p. 77 (§4 problems)

---

tidy | SemBr | en | review

Footnotes

1. 2022. MATH1011: Multivariable calculus, pp. 65–66 ↩

2. 2016. Calculus, pp. 1011–1012 ↩