An optimal control theory for nonlinear optimization

Abstract

The Karush–Kuhn–Tucker conditions for a given nonlinear programming problem are generated as the transversality conditions of an optimal control problem. The directional derivatives of the objective- and constraint-functions supply the vector fields for the optimal control problem with the search vector as the control variable. Zero-Hamiltonian trajectories along the steepest descent of control Lyapunov functions provide optimal optimization algorithms. The optimality of the algorithm also depends upon the choice of a metric for the finite dimensional control space. Many well-known algorithms – such as Newton’s method, the first-order Lagrangian method, the steepest descent method and Richardson’s method, to name a few – are derived by minimizing the Lie derivative of a quadratic control Lyapunov function. Merit functions in optimization may also be generated using the concept of a control Lyapunov function. These results suggest that optimal control principles hold the potential for a unified theory for optimization.