A single network approximate dynamic programming based constrained optimal controller for nonlinear systems with uncertainties

Approximate dynamic programming formulation implemented with an Adaptive Critic (AC) based neural network (NN) structure has evolved as a powerful alternative technique that eliminates the need for excessive computations and storage requirements needed for solving the Hamilton-Jacobi-Bellman (HJB) equations. A typical AC structure consists of two interacting NNs. In this paper, a novel architecture, called the Cost Function Based Single Network Adaptive Critic (J-SNAC) is used to solve control-constrained optimal control problems. Only one network is used that captures the mapping between states and the cost function. This approach is applicable to a wide class of nonlinear systems where the optimal control (stationary) equation can be explicitly expressed in terms of the state and costate variables. A non-quadratic cost function is used that incorporates the control constraints. Necessary equations for optimal control are derived and an algorithm to solve the constrained-control problem with J-SNAC is developed. Benchmark nonlinear systems are used to illustrate the working of the proposed technique. Extensions to optimal control-constrained problems in the presence of uncertainties are also considered.