THE USE OF LAGRANGE MULTIPLIER’S METHOD IN SOLVING THE CONSTRAINED OPTIMIZATION PROBLEM

ABSTRACT.

This project proposes the use of Lagrange multiplier’s method in solving the constrained optimization problem. This problem arise in many applications to various fields, such as economics, physics, geometry e.t.c.It has been shown under condition(s), that Lagrange multipliers method can be used to find the extrema of a function.

CHAPTER ONE----------------------------------------------------------------------------------------1

1.0    Introduction-------------------------------------------------------------------------------------1

1.1  Significance--------------------------------------------------------------------------------------2

1.2  Research question-----------------------------------------------------------------------------3

1.3  Aims and objectives---------------------------------------------------------------------------4

1.4  scope/Limitations------------------------------------------------------------------------------5

1.5  defination of terms----------------------------------------------------------------------------5

CHAPTER TWO---------------------------------------------------------------------------------------8

2.0    Literature review-------------------------------------------------------------------------------8

2.1  Lagrange Multipliers method-------------------------------------------------------------10

2.2 Mathematical proof of Lagrange Multipliers method-------------------------------16

2.3 Geometric explanation of the method of Lagrange multiplier’s method-------18

CHAPTER THREE------------------------------------------------------------------------------------21

3.0    Applications of lagrange multiplier------------------------------------------------------21

3.1 Economic applications of lagrange multiplier------------------------------------------22

3.2 Application to power system economic operation-----------------------------------27

3.3 Extrema, subject to one constraint-------------------------------------------------------29

3.4 constrained extrema of quadratic form-------------------------------------------------33

3.5 Extrema subject to two constraints------------------------------------------------------35

CHAPTER FOUR-------------------------------------------------------------------------------------40

4.0    solved problems and discussions of results--------------------------------------------40

4.1 problems with Economic Application----------------------------------------------------40

4.2 problems with Geometric application---------------------------------------------------43

4.3 problems with two constraints------------------------------------------------------------47

4.4 Summary----------------------------------------------------------------------------------------51

4.5 Conclusion--------------------------------------------------------------------------------------52

References-------------------------------------------------------------------------------------------53

CHAPTER ONE

1.0      INTRODUCTION

Lagrange multipliers are very useful technique in multivariable calculus. One of the most common problem in calculus is that of finding maxima and minima (in general, extrema) of a function, but is often difficult to find a closed form for the function being extremize, such difficulties often arise when our wishes to maximize or minimize a function subject to fixed outside conditions or constraint. The method of Lagrange multiplier is a powerful tool for solving this class of problems without the need to explicitly solve the condition and use them to eliminate extra variables. In general, Lagrange multiplier are useful when some of the variables in the simplest description of a problem are made redundant by the constraint.

Optimization problems, which seek to minimize or maximize a real functional play an important role in the real world. It can be classified into unconstrained optimization problems and constrained optimization problems. Many practical uses in science, engineering, economics, or even in our everyday life can be formulated as constrained optimization problems, such as: [1] The minimization of the energy of a particle in physics; [2] How to maximize the profit of the investments in economics.

In unconstrained problems, the stationary point’s theory gives the necessary condition to find the extreme points of the objective function f(x1,….xn) the stationary points are the points where the gradient f is zero. i.e. each of the partial derivatives is zero. All the variables in f(x1,….xn) are independent, so they can be arbitrarily set to seek the extreme of F. However when it comes to the constrained optimization problems, the arbitration of the variables does not exist. The constrained optimization problems can be formulated into the standard forms as:

 

Min. f(x1,….xn)

Subject to: G (x1,….xn) = 0    

                      H (x1,….xn) = 0

Where: G, H are function vectors. The variables are restricted to the feasible region, which refers to the points satisfying the constraints.

Substitution is an intuitive method to deal with optimization problem. But it can only apply to the equality constrained optimization problems where it is difficult to get the explicit expressions for the variables needed to be eliminated in the objective function.

The Lagrange multipliers method, named after Joseph Louis Lagrange, provide an alternative method for the constrained non-linear optimization problems. It can help deal with both equality constraints

 

1.1                             SIGNIFICANCE OF THE STUDY

On the use of Lagrange. Multiplier compatible modes for controlling accuracy and stability of mixed shell finite elements classical hybrid type formulations for shell finite elements can be developed from Helinger – Reissner Virational Principle combined with the use of stress basis functions which satisfy the homogeneous equilibrium equation. The technique can provide accurate stresses, although implementation for the governing equations and the difficulty of satisfying coordinate invariance.

The introduction of Lagrange multipliers provides a means of gaining control over this balance. It is shown in the context of Mindlin Kinematics that the use of element-based compactable Lagrange multipliers with local bubble basis function can lead to accurate stresses, including: transverse shear stresses, and the complete elimination of transverse shear locking.

The Lagrange multiplier  has significance in economics as well. If you are maximizing profit subject to a limited resource,  is that resources marginal value (some times called the “Shadow Price” of the resources). Specifically, the value of the function F(p) changes if we change the constraint .

To express this mathematically; write the constraint in the form

g(p) = g(x, y) = c for some constant C. you can solve most constrained optimization problems by the method of Lagrange multipliers without actual obtaining a numerical value for the numerical value for Lagrange multiplier . In some problems   has the following useful interpretation.

 = , where M is the maximum value subject to K = g(x, y). Then   is rate of change of M with respect to K. Hence    change in M resulting from a

1-unit increase in K

 

1.2 RESEARCH QUESTIONS

In this research question, the emphasis is on generating a unique question and ten synthesizing source into a coherent essay that supports the method of Lagrange multiplier and its applications.

The following question(s) raised from my curiosity is to help further my research on this topic:

 

WHY DOES THE METHOD OF LAGRANGE MULTIPLIER WORKS?

Although a rigorous explanation of why the method of Lagrange multiplier involves advanced ideas beyond the cop of this text, there is a rather simple geometric argument that I found convincing.

This argument depends on the fact that for the level curve F((x, y) = c, the slope is given by:

              = -     

This result is true for any level curve of a function F whose partial derivatives exist (provided fy  0) now consider the constrained optimization problem: maximize f(x, y) subject to g(x, y) =k

Geometrically, this means you must find the highest level curve of F that intersects the constraint curve g(x,y)=k. The critical intersection will occur at a point where the constraint curve is tangent to a level curve; that is, where the slop of the constraint curve f(x,y)=c.

According to the formula stated above, we have slope of constraint curve

 = slope of level curve

-   =  -

Or equivalently,   

           =

If we let devote this common ration, then

              =  and   =  

From which we get the first two Lagrange equations

fx =  gx and fy =  gy

The third Lagrange equation g(x, y)=K is simply a statement of the fact of the point of tangency actually lies on the constraint curve.

 

1.3 AIMS AND OBJECTIVES

The aim and objectives of this research is:

  1. To find the extrema of a function subject to a fixed constraint through an analytical investigation of Joseph Louis Lagrange’s (1736-1813) work; referred to as Lagrange multiplier’s method.
  2. To see it (Lagrange multiplier) applications in a variety of field, including economics etc.
  3. To know the method of Lagrange multiplier
  4. To minimize/maximize a multivariable function subject to one or multiple constraint
  5. To understand how the method of Lagrange multipliers can be used to find absolute maximums and absolute minimums of a function over a close region
  6. To understand the application of Lagrange multipliers on economic

1.4 SCOPE/LIMITATIONS

This research on finding the extrema of a function of a constrained optimization problem having one or multiple variables based on the method of Lagrange multipliers. The method, ranging from its mathematical proof, geometric explanation to it economic applications will be used to carryout this research.

The objective in optimization is to find the minima or maxima of a cost function f  that depends upon n variables . A minima (or maxima) solution must satisfy necessary and sufficient conditions.

 

1.5 DEFINITION OF TERMS

  1. LAGRANGIAN: The Lagrangian L, of a dynamical system is a mathematical function that summarizes the dynamics of the system or a function that describes the state of a dynamic system in terms of position coordinates and their time derives.
  2. MAXIMA AND MINIMA: The maxima and minima (the plural of maximum and minimum) of a function are the largest and smallest value of the function, either within a given range or on the entire domain of a function.
  3. EXTREMA: The extrema (the plural of extremum) is a value in the domain of a given function at which the function attains a maximum or a minimum value.
  4. CONSTRAINT: A constraint is a condition of an optimization problem the solution must satisfy
  5. FEASIBLE SET: The set of candidate solutions that satisfy all the constraints.
  6. OPTIMIZATION: This is the selection of a best element (with regard to some criteria) from some set of available alternatives. More generally, optimization includes finding “best available” values of some objectives function given a defined domain (or a set of constraints).
  7. NON LINEAR SYSTEM: A non linear system of equations is a set of simultaneous equations in which the unknown functions appear as variables of a polynomial of degree higher than one.
  8. VARIABLE: A symbol for a number we don’t know yet it is usually a letter like x or y. e.g. x + 2 = 6, x is the variable.
  9. EXPLICIT: (of a function) having the dependent variable expressed directly in terms of the independent variables e.g. y=3x + 4
  10. DERIVATIVE: A function which gives the slope of a curve; that is the slope of the tangent to a function. The derivative of a function f at a point x is commonly written as f’(x).

Mathematically, f ‘(a) =  =       f ’(x)=          

  1. GEOMETRY: The area of mathematics that deals with points lines, shapes, planes and figures, and examines their properties, measurement and mutual relations in space.
  2. GRADIENT: The gradient in vector analysis, is a vector operators denoted  and sometimes called de/ or nabla. It is most often applied to a real function of three variable f(U1, U2, U3) and may be denoted, f = grad(f)
  3. STATIONARY POINT: A point on a curve where the gradient is zero. A stationary point may be a minimum, maximum or inflection point
  4. SADDLE POINT: A point at which a function of two variables has partial derivatives equal to zero but at which the function has neither a maximum nor a minimum value.
  5. LAMBDA ( ): The 11th letter of Greek alphabet, which is used to denote the variable, (2) called Lagrange multiplier
  6. CONTOUR CURVES: The level set of f at value c (for functions f(x, y) of two variables) is the set of all points (x, y) such that f(x, y) = c