On optimality of passivity based controllers in discrete-time (original) (raw)

Contents lists available at ScienceDirect

Systems & Control Letters

journal homepage: www.elsevier.com/locate/sysconle

Review

On optimality of passivity based controllers in discrete-time

Salvatore Monaco a{ }^{a}, Dorothée Normand-Cyrot b, { }^{\text {b, }} *

a{ }^{a} Dipartimento di Ingegneria Informatica, Automatica e Gestionale ‘Antonio Ruberti’, Sapienza Università di Roma, via Ariosto 25, 00185 Roma, Italy
b{ }^{\mathrm{b}} Laboratoire des Signaux et Systèmes, CNRS-Supelec, Plateau de Moulon, 91190 Gif-sur-Yvette, France

ARTICLE INFO

Article history:
Received 6 June 2013
Received in revised form
16 October 2014
Accepted 16 October 2014
Available online 20 November 2014

Keywords:
Nonlinear discrete-time systems
Optimal control
Nonlinear stabilization
Passivity based control
Lyapunov design

Contents

  1. Introduction… 117
  2. Problem settlement… 118
  3. Optimal control… 119

3.1. The quadratic case … 119
3.2. Inverse optimal control… 120
4. Optimality and uu-average dissipativity … 120
5. Some examples and constructive cases… 121
5.1. The linear case… 122
5.2. The case G(x,u)=G(x)G(x, u)=G(x)… 122
5.3. The input-affine case … 122
6. Conclusions… 123
References… 123

1. Introduction

Connections between optimality and passivity based controllers are well established in the continuous-time context [1-4]. Following these lines we investigate the link between optimality and passivity-like properties in discrete time. It is shown that the feedback control law which minimizes a certain criterion defines a damping strategy which recalls the passivity based controller studied in [5]. In that work, making use of the representation of discrete-time dynamics as coupled difference and dif-

[1]ferential equations [6,22], the authors introduced the notion of uu average passivity which brings to the discrete-time counterparts of passifying output mapping and negative output feedback stabilizing controller. In the present work we show that, starting from the characterization of the associated Hamilton-Jacobi-Bellman (HJB) equation, an implicit description of the optimal controller can be obtained for a quite general class of nonlinear discrete-time dynamics. The link with uu-average damping control is then discussed.

Passivity concepts and nonlinear stabilization in discrete time have been addressed in several papers in the last twenty years under specific restrictions on the nonlinearities [7-11] or for sampled-data dynamics in an approximated context [12-14].

Despite the interesting connections between optimality and passivity established in continuous time for the class of inputaffine dynamics (e.g. [3]), only particular results are available in discrete time where the investigation is restricted to quadratic

nonlinearities [8,15][8,15]. The present paper, which extends results presented in [16], is a first attempt to deal with more general nonlinear discrete-time dynamics.

Apart from the definition of passivity itself [17,5], the major difficulty in discrete time is linked to the fact that only an implicit characterization of the solution can be given. Moreover the controller and the HJB equation are interlaced. Thus, to better clarify the optimality properties of a stabilizing controller becomes in discrete time a fundamental step for the design.

In the present paper, optimality of a feedback controller is characterized in terms of output feedback average passivity with respect to a dummy output depending on the storage function or control Lyapunov function (CLF). On these bases, a new negative output feedback is shown to be inverse optimal for a specific cost for a wide class of nonlinear discrete-time dynamics. Some particular cases for which explicit controllers can be given are discussed to illustrate the results in a comparative setting with respect to both the continuous-time context and specific results obtained in discrete-time.

The paper is organized as follows. The problem is set in Section 2 after recalling the differential/difference framework proposed to represent discrete-time dynamics together with the notion of average dissipativity. Optimal and inverse optimal control solutions are described in Section 3. Section 4 discusses the connections with uu-average dissipativity concepts. Some constructive cases are treated in Section 5 making references to specific state-space structures so recovering also the input-affine case studied in the literature [10,8,15][10,8,15]. For the sake of notational simplicity, the study is limited to single-input discrete-time dynamics.

2. Problem settlement

As in [6], the following couple of equations is used to describe a nonlinear discrete-time single-input dynamics Σd\Sigma_{d}
x+=F0(x)x^{+}=F_{0}(x)
∂x+(u)∂u=G(x+(u),u)\frac{\partial x^{+}(u)}{\partial u}=G\left(x^{+}(u), u\right) \quad with x+(0)=x+x^{+}(0)=x^{+}
where F0F_{0} is a RnR^{n}-valued smooth map and G(.,u)G(., u) is a vector field on RnR^{n}, parameterized by u∈Uu \in U and assumed complete. For any fixed (xk,uk)\left(x_{k}, u_{k}\right) at time k,(1)k,(1) gives the initial condition of (2),xk+(0)=(2), x_{k}^{+}(0)= F0(xk)F_{0}\left(x_{k}\right), and by integrating (2) one gets
x+(uk)=xk+(0)+∫0ukG(x+(v),v)dvx^{+}\left(u_{k}\right)=x_{k}^{+}(0)+\int_{0}^{u_{k}} G\left(x^{+}(v), v\right) \mathrm{d} v
so recovering the usual representation of a discrete-time dynamics as a map
xk+1=x+(uk)=F(xk,uk)x_{k+1}=x^{+}\left(u_{k}\right)=F\left(x_{k}, u_{k}\right)
with F(xk,0)=F0(xk)=xk+(0)F\left(x_{k}, 0\right)=F_{0}\left(x_{k}\right)=x_{k}^{+}(0). On the other side, a smooth map describing a discrete-time dynamics can be represented in the form (1)-(2) provided ∂F(x,u)∂t=G(F(x,u),u)\frac{\partial F\left(x, u\right)}{\partial t}=G(F(x, u), u) for a suitable G(.,u)G(., u). We assume in the sequel that x=0x=0 is an equilibrium for Σd\Sigma_{d}, i.e. F0(0)=0F_{0}(0)=0.

Given (xk,uk)\left(x_{k}, u_{k}\right) at time k∈Nk \in N, we consider the usual cost functional
J(xk)=∑n≥kI(xn)+unγR(xn)unJ\left(x_{k}\right)=\sum_{n \geq k} I\left(x_{n}\right)+u_{n}^{\gamma} R\left(x_{n}\right) u_{n}
where I:RnrightarrowR+isapositivesemidefinitefunctionisapositivesemidefinitefunction(I(x)geq0)I: R^{n} \rightarrow R_{+}isapositivesemidefinitefunctionis a positive semidefinite function (I(x) \geq 0)I:RnrightarrowR+isapositivesemidefinitefunctionisapositivesemidefinitefunction(I(x)geq0) and R:Rn→RR: R^{n} \rightarrow R a positive definite weight (uγR(x)u>0,∀x≠0)\left(u^{\gamma} R(x) u>0, \forall x \neq 0\right). The task of the optimal control strategy is to find a feedback u(x)u(x) which:

The value of JJ when computed at its minimum is a function of the initial condition x0x_{0} only; it defines what it is called the optimal value function. The optimal control value is denoted as usual by u∗u^{*}. In the sequel Jk,Ik,RkJ_{k}, I_{k}, R_{k} stand for J(xk),I(xk),R(xk)J\left(x_{k}\right), I\left(x_{k}\right), R\left(x_{k}\right) when no confusions are possible.

Some manipulations over the trajectories associated with dynamics (1)-(2) are instrumental. The expansion in uu of G(.,u)G(., u) as G(.,u)=G1(.)+∑i≥1ui2Gi+1(.)G(., u)=G_{1}(.)+\sum_{i \geq 1} \frac{u^{i}}{2} G_{i+1}(.) defines a family of control vector fields on RnR^{n}, the Gi∗G_{i}^{*}, which characterize the geometric structure of the flow associated with the differential equation (2) (see [6] for further details). The integration of (2) with initial condition xk+(0)=F0(xk)x_{k}^{+}(0)=F_{0}\left(x_{k}\right) gives

xk+1:=x+(uk)=xk+(0)+∫0ukG(x+(v),v)dv=eukψ(.,uk)xk+(0)\begin{aligned} x_{k+1}:=x^{+}\left(u_{k}\right) & =x_{k}^{+}(0)+\int_{0}^{u_{k}} G\left(x^{+}(v), v\right) \mathrm{d} v \\ & =e^{u_{k} \psi(., u_{k})} x_{k}^{+}(0) \end{aligned}

where the exponent series uψ(.,u)u \psi(., u) of the flow euψ(.,u)e^{u \psi(., u)} associated with G(.,u)G(., u) admits the asymptotic expansion
uψ(.,u)=uG1+u22G2+u33!(G3+12[G1,G2])+O(u4)u \psi(., u)=u G_{1}+\frac{u^{2}}{2} G_{2}+\frac{u^{3}}{3!}\left(G_{3}+\frac{1}{2}\left[G_{1}, G_{2}\right]\right)+O\left(u^{4}\right)
in terms of the vector fields GiG_{i} and their successive Lie brackets. The exponential form (4) can be further developed so recovering the asymptotic expansion in powers of uu of the discrete-time mapping F(xk,uk)F\left(x_{k}, u_{k}\right). One gets

ParseError: KaTeX parse error: Expected '}', got '&' at position 189: …_{F_{0}(x)} \\ &̲ =F_{0}(x)+\lef…

Analogously to (4), given any mapping H:Rn→RH: R^{n} \rightarrow R, one computes H(x+(u))H\left(x^{+}(u)\right) through integration with respect to uu at x+(0)x^{+}(0), as
H(x+(u))=H(x+(0))+∫0u LG(.,v)H(x+(v))dvH\left(x^{+}(u)\right)=H\left(x^{+}(0)\right)+\int_{0}^{u} \mathrm{~L}_{G(., v)} H\left(x^{+}(v)\right) \mathrm{d} v
with LG(.,v)H(x+(v))=LG1H∣x+(0)+v( LG12H+LG2H)∣x+(0)+O(v2)\mathrm{L}_{G(., v)} H\left(x^{+}(v)\right)=\left.\mathrm{L}_{G_{1}} H\right|_{x^{+}(0)}+\left.v\left(\mathrm{~L}_{G_{1}}^{2} H+\mathrm{L}_{G_{2}} H\right)\right|_{x^{+}(0)}+\left.O\left(v^{2}\right)\right.. In a more compact form, exploiting the flow associated with the solution to the differential equations (2), one gets
ParseError: KaTeX parse error: Expected 'EOF', got '\right' at position 190: …right.} \cdot H\̲r̲i̲g̲h̲t̲|_{F_{0}(x)}.
If HH denotes the output mapping, one has yk+1=H(xk+(uk))y_{k+1}=H\left(x_{k}^{+}\left(u_{k}\right)\right) and for the unforced evolution, y∣k+1=H(F0(xk))=H(xk+(0))y_{\mid k+1}=H\left(F_{0}\left(x_{k}\right)\right)=H\left(x_{k}^{+}(0)\right).

The following notation will be used in the sequel to characterize the terms of order two in the expansion of H(x+(u))H\left(x^{+}(u)\right) in powers of uu :

ParseError: KaTeX parse error: Expected & or \\ or \cr or \end at position 59: …_{H_{0}(x)}^{+}}̲+\left.u L_{G_{…

with

RH(x+(u),u):=∂LG(.,u)H(x+(u))∂u=LG(.,u)2H(x+(u))+LuG,u)2uH(x+(u))\begin{aligned} \mathcal{R}_{H}\left(x^{+}(u), u\right) & :=\frac{\partial \mathrm{L}_{G(., u)} H\left(x^{+}(u)\right)}{\partial u} \\ & =\mathrm{L}_{G(., u)}^{2} H\left(x^{+}(u)\right)+\mathrm{L} \frac{\left.u G, u\right)}{2 u} H\left(x^{+}(u)\right) \end{aligned}

and RH(x+(0),0):=LG12H(x+(0))+LG2H(x+(0))\mathcal{R}_{H}\left(x^{+}(0), 0\right):=\mathrm{L}_{G_{1}}^{2} H\left(x^{+}(0)\right)+\mathrm{L}_{G_{2}} H\left(x^{+}(0)\right).
In the previous equalities, IdI_{d} denotes the identity function, LG\mathrm{L}_{G} (equivalently GG when no ambiguity is possible) denotes the Lie derivative along the vector field G(x)G(x), and ∣x\left.\right|_{x} (equivalently (x)) denotes evaluation at xx of any function. Completeness of the vector field G(.,u)G(., u) for uu in a neighborhood of 0 guarantees the existence of an exponential form representation of the flow (4) associated with the differential equation (2). The maps which are parameterized by uu admit asymptotic expansions in powers of uu. When series expansions are manipulated, the notation r∈O(up)r \in O\left(u^{p}\right) indicates that the remaining terms rr are in uu at order pp at least or are dominated when uu goes to 0 by upu^{p}, i.e. lim⁡u→0rup\lim _{u \rightarrow 0} \frac{r}{u^{p}} is bounded.

3. Optimal control

In this section, the properties of the optimal solution are investigated. With respect to the existing results, the proposed formulation provides an implicit characterization of the optimal control solution with practical advantages when an explicit formula can be deduced from (8). This aspect is addressed in Remark 3.1 and illustrated through particular cases in Sections 3.1 and 5.

Theorem 3.1. Given the discrete-time dynamics (1)-(2), suppose that there exists a C2C^{2} positive semidefinite function V(x)V(x) satisfying for any x∈Rnx \in R^{n} the HJB equation

V(F0(x))−V(x)+l(x)+∫0u∗LG(⋅,u)V∣x+(u)∣du+(u∗)TR(x)u∗=0\begin{aligned} & V\left(F_{0}(x)\right)-V(x)+l(x)+\int_{0}^{u^{*}} L_{G(\cdot, u)} V\left|{ }_{x^{+}(u)}\right| d u \\ & \quad+\left(u^{*}\right)^{T} R(x) u^{*}=0 \end{aligned}

where u∗=u∗(x)u^{*}=u^{*}(x) is a stabilizing control implicitly defined by

u∗=−12R−1(x)LG(⋅,u∗)V(x+(u∗))u^{*}=-\frac{1}{2} R^{-1}\left(x\right) L_{G(\cdot, u^{*})} V\left(x^{+}\left(u^{*}\right)\right)

with RR satisfying the inequality

RV(x+(u∗),u∗)+2R(x)>0\mathscr{R}_{V}\left(x^{+}\left(u^{*}\right), u^{*}\right)+2 R(x)>0

Then, u∗u^{*} is the optimal stabilizing control which minimizes the cost (3) and V(x)V(x) is the optimal value function.

Proof. Let H(xk,uk)\mathscr{H}\left(x_{k}, u_{k}\right) be the discrete-time Hamiltonian associated with the cost functional (3); i.e.
H(xk,uk)=lk+ukTRkuk+V(xk+1)−V(xk)\mathscr{H}\left(x_{k}, u_{k}\right)=l_{k}+u_{k}^{\mathrm{T}} R_{k} u_{k}+V\left(x_{k+1}\right)-V\left(x_{k}\right)
which can be rewritten, in view of the integral form (4), as
Hk=lk+ukTRkuk+V(xk+(0))−V(xk)+∫0uk LG(⋅,v)V∣xk+(v)∣dv\mathscr{H}_{k}=l_{k}+u_{k}^{\mathrm{T}} R_{k} u_{k}+V\left(x_{k}^{+}(0)\right)-V\left(x_{k}\right)+\int_{0}^{u_{k}} \mathrm{~L}_{G(\cdot, v)} V\left|{ }_{x_{k}^{+}(v)}\right| d v.
A direct application of the principle of optimality [18] for the infinite horizon case says that V(xk)V\left(x_{k}\right) goes to an invariant value satisfying
V(xk)=min⁡uk(lk+ukTRkuk+V(xk+1))V\left(x_{k}\right)=\min _{u_{k}}\left(l_{k}+u_{k}^{\mathrm{T}} R_{k} u_{k}+V\left(x_{k+1}\right)\right).
The control uku_{k} which minimizes Hk\mathscr{H}_{k} satisfies the condition

Hu′(xk,uk)=∂H(xk,u)∂u∣uk=LG(⋅,uk)V∣xk+(uk)∣+2Rkuk=0\left.\mathscr{H}_{\mathrm{u}}^{\prime}\left(x_{k}, u_{k}\right)=\frac{\partial \mathscr{H}\left(x_{k}, u\right)}{\partial u}\right|_{u_{k}}=\mathrm{L}_{G(\cdot, u_{k})} V\left|{ }_{x_{k}^{+}}\left(u_{k}\right)\right|+2 R_{k} u_{k}=0

so getting the implicit solution (8). The condition H(xk,uk∗)=0\mathscr{H}\left(x_{k}, u_{k}^{*}\right)=0 restitutes the HJB equation (7) since by definition

H(xk,uk∗)=lk+V(F0(xk))−V(xk)+∫0uk∗ LG(⋅,u)V∣xk+(u)∣du+(uk∗)TRkuk∗\begin{aligned} \mathscr{H}\left(x_{k}, u_{k}^{*}\right)= & l_{k}+V\left(F_{0}\left(x_{k}\right)\right)-V\left(x_{k}\right) \\ & +\int_{0}^{u_{k}^{*}} \mathrm{~L}_{G(\cdot, u)} V\left|{ }_{x_{k}^{+}(u)}\right| \mathrm{d} u+\left(u_{k}^{*}\right)^{T} R_{k} u_{k}^{*} \end{aligned}

Substituting uku_{k} with uk∗+vku_{k}^{*}+v_{k} into the cost functional (3) and applying (7) and (10), one gets the following chain of equalities

J(xk)=∑n≥k(H(xn,un∗+vn)−V(xn+1)+V(xn))=∑n≥kH(xn,un∗+vn)+V(xk)−lim⁡k→∞V(xk)\begin{aligned} J\left(x_{k}\right) & =\sum_{n \geq k}\left(\mathscr{H}\left(x_{n}, u_{n}^{*}+v_{n}\right)-V\left(x_{n+1}\right)+V\left(x_{n}\right)\right) \\ & =\sum_{n \geq k} \mathscr{H}\left(x_{n}, u_{n}^{*}+v_{n}\right)+V\left(x_{k}\right)-\lim _{k \rightarrow \infty} V\left(x_{k}\right) \end{aligned}

with

H(xn,un∗+vn)=H(xn,un∗)+vn LG(⋅,un∗))V∣xn+(un∗))+vnTRnvn+2vnTRnun∗+∫0vn∫0wnRV(xn+(un∗+θn),un∗+θn)dθn dwn\begin{aligned} & \left.\mathscr{H}\left(x_{n}, u_{n}^{*}+v_{n}\right)=\mathscr{H}\left(x_{n}, u_{n}^{*}\right)+v_{n} \mathrm{~L}_{G(\cdot, u_{n}^{*})}\right) V\left|{ }_{x_{n}^{+}}\left(u_{n}^{*}\right)\right)+v_{n}^{T} R_{n} v_{n} \\ & \quad+2 v_{n}^{T} R_{n} u_{n}^{*}+\int_{0}^{v_{n}} \int_{0}^{w_{n}} \mathscr{R}_{V}\left(x_{n}^{+}\left(u_{n}^{*}+\theta_{n}\right), u_{n}^{*}+\theta_{n}\right) \mathrm{d} \theta_{n} \mathrm{~d} w_{n} \end{aligned}

and from (6)

∂2H(xn,u)∂2u∣un∗=∂LG(⋅,u)V(x+(u))∂u∣un∗+2Rn=LG(⋅,un∗)2V∣xn+(un∗)+L∂G(⋅,u)∂u∣un∗V∣xn+(un∗)+2Rn=RV(xn+(un∗),un∗)+2Rn\begin{aligned} \left.\frac{\partial^{2} \mathscr{H}\left(x_{n}, u\right)}{\partial^{2} u}\right|_{u_{n}^{*}} & =\left.\frac{\partial \mathrm{L}_{G(\cdot, u)} V\left(x^{+}(u)\right)}{\partial u}\right|_{u_{n}^{*}}+2 R_{n} \\ & =\mathrm{L}_{G(\cdot, u_{n}^{*})}^{2} V\left|{ }_{x_{n}^{+}}\left(u_{n}^{*}\right)+\mathrm{L}_{\frac{\partial G(\cdot, u)}{\partial u}}\right|_{u_{n}^{*}} V\left|{ }_{x_{n}^{+}}\left(u_{n}^{*}\right)+2 R_{n}\right. \\ & =\mathscr{R}_{V}\left(x_{n}^{+}\left(u_{n}^{*}\right), u_{n}^{*}\right)+2 R_{n} \end{aligned}

After substitution, one gets in x0x_{0}

J(x0)=∑k≥0∫0vk∫0wkRV(xk+(uk∗+θk),uk∗+θk)dθk dwk+∑k≥0vkTRkvk+V(x0)−lim⁡k→∞V(xk)\begin{aligned} J\left(x_{0}\right)= & \sum_{k \geq 0} \int_{0}^{v_{k}} \int_{0}^{w_{k}} \mathscr{R}_{V}\left(x_{k}^{+}\left(u_{k}^{*}+\theta_{k}\right), u_{k}^{*}+\theta_{k}\right) \mathrm{d} \theta_{k} \mathrm{~d} w_{k} \\ & +\sum_{k \geq 0} v_{k}^{\mathrm{T}} R_{k} v_{k}+V\left(x_{0}\right)-\lim _{k \rightarrow \infty} V\left(x_{k}\right) \end{aligned}

Since one minimizes (10) over the controls which achieve lim⁡k→∞\lim _{k \rightarrow \infty} xk=0x_{k}=0 and lim⁡k→∞V(xk)=0\lim _{k \rightarrow \infty} V\left(x_{k}\right)=0, one obtains
J(x0)=12∑k≥0vkT(RV(xk+(uk∗+vk),uk∗+vk)+2Rk)vk+V(x0)J\left(x_{0}\right)=\frac{1}{2} \sum_{k \geq 0} v_{k}^{\mathrm{T}}\left(\mathscr{R}_{V}\left(x_{k}^{+}\left(u_{k}^{*}+v_{k}\right), u_{k}^{*}+v_{k}\right)+2 R_{k}\right) v_{k}+V\left(x_{0}\right).
Because of (9), a local minimum is reached with v=0v=0, which proves that u∗u^{*} is the optimal control and V(x0)V\left(x_{0}\right) the optimal value. ◃\triangleleft
Remark 3.1. (7) and (8) provide an implicit characterization of the optimal control problem solution because an explicit formula for u∗u^{*} cannot be obtained in general from (8). To overcome such an obstruction, the idea proposed in [19] of assuming a fixed structure to the solution of an implicit equation could be pursued by assuming u∗u^{*} of the form
u∗(x)=−γ(x)LG1V(x+(0))=−γ(x)∂V(x)∂x∣F0(x)G1(F0(x))u^{*}(x)=-\gamma(x) \mathrm{L}_{G_{1}} V\left(x^{+}(0)\right)=-\gamma(x)\left.\frac{\partial V(x)}{\partial x}\right|_{F_{0}(x)} G_{1}\left(F_{0}(x)\right)
with γ(x)\gamma(x) a suitable gain function to be computed.
Remark 3.2. The condition of positivity (9) can be assured by a proper choice of the weighting function R(x)R(x) noting that at x=0x=0, x0T(0)=F0(0)=0x_{0}^{\mathrm{T}}(0)=F_{0}(0)=0 and (9) is satisfied
LG12V(0)+2R(0)=G1T(0)∂2V∂x2∣0G1(0)+2R(0)>0)\mathrm{L}_{G_{1}}^{2} V(0)+2 R(0)=\left.G_{1}^{\mathrm{T}}(0)\left.\frac{\partial^{2} V}{\partial x^{2}}\right|_{0} G_{1}(0)+2 R(0)>0\right)
because VV is positive semidefinite with V(0)=0,∂V∂x∣0=0V(0)=\left.0, \frac{\partial V}{\partial x}\right|_{0}=0 and ∂2V∂x2∣0≥0\left.\frac{\partial^{2} V}{\partial x^{2}}\right|_{0} \geq 0.

3.1. The quadratic case

The implicit characterization of the optimal solution given in Theorem 3.1 provides a theoretical result which deserves more practical solutions under specific assumptions. In the concerned literature starting from [9], a commonly adopted assumption is quadraticity in uu of the one step forward difference V(x+(u))−V\left(x^{+}(u)\right)- V(x)V(x). In the sequel the result of Theorem 3.1 is illustrated making reference to an Hamiltonian function (10) which is quadratic in uu either thanks of truncations in 0(u2)0\left(u^{2}\right) of its series expansion or as a consequence of suitable assumptions on the system structure. In the present formulation (5), this corresponds to assume the existence of a function V≥0V \geq 0 such that for any xx in RnR^{n} :
A1: ∂LG(⋅,u)V(x+(u))∂u:=RV(x+(0),0)\frac{\partial \mathrm{L}_{G(\cdot, u)} V\left(x^{+}(u)\right)}{\partial u}:=\mathscr{R}_{V}\left(x^{+}(0), 0\right).
Under A1, both the one step forward difference V(x+(u))−V(x)V\left(x^{+}(u)\right)-V(x) and the Hamiltonian are quadratic in uu and H(x,u)\mathscr{H}(x, u) rewrites as:
H(x,u)=V(x+(0))−V(x)+l+u LG1V(x+(0))+12uTD(x+(0))u\mathscr{H}(x, u)=V\left(x^{+}(0)\right)-V(x)+l+u \mathrm{~L}_{G_{1}} V\left(x^{+}(0)\right)+\frac{1}{2} u^{\mathrm{T}} D\left(x^{+}(0)\right) u
with D(x+(0)):=RV(x+(0),0)+2R(x)D\left(x^{+}(0)\right):=\mathscr{R}_{V}\left(x^{+}(0), 0\right)+2 R(x).

It can be easily verified that under assumption A1:

u∗=−LC1V(F0(x))D(F0(x))u^{*}=-\frac{\mathrm{L}_{\mathrm{C}_{1}} V\left(F_{0}(x)\right)}{D\left(F_{0}(x)\right)}

which exhibits the form (12) given in Remark 3.1;

V(F0(x))−V(x)+l(x)−12 LC1V(F0(x))T LC1V(F0(x))D(F0(x))=0V\left(F_{0}(x)\right)-V(x)+l(x)-\frac{1}{2} \frac{\mathrm{~L}_{\mathrm{C}_{1}} V\left(F_{0}(x)\right)^{\mathrm{T}} \mathrm{~L}_{\mathrm{C}_{1}} V\left(F_{0}(x)\right)}{D\left(F_{0}(x)\right)}=0

According to the previous comments, Theorem 3.1 specifies as follows.

Corollary 3.1. Given the discrete-time dynamics (1)-(2), suppose that there exists a positive semidefinite function V(x)V(x) which satisfies A1. If V(x)V(x) verifies the HJB equation (14) and u∗u^{*} in (13) is a stabilizing control, then it is the optimal stabilizing feedback which minimizes the cost (3) and V(x0)V\left(x_{0}\right) is the optimal value.

3.2. Inverse optimal control

The interest in pursuing an inverse approach is even more evident in discrete-time, due to the difficulty to explicitly express u∗u^{*}. As well known, in this case, a stabilizing feedback is designed first and shown to be optimal for a certain cost. Discrete-time inverse optimal control is discussed in [15] for input-affine dynamics in relation with passivation. The following proposition provides, in the formalism adopted, a simple characterization which is instrumental to the results stated in the next section.

Proposition 3.1. Consider the discrete-time dynamics (1)-(2). Assume that there exists a positive semidefinite function VV with V(0)=V(0)= 0 which satisfies for all x∈Rnx \in R^{\mathrm{n}}
V(F0(x))−V(x)+∫0u∗ LGl,uV∣x+(u)du+(u∗)TR(x)u∗≤0V\left(F_{0}(x)\right)-V(x)+\int_{0}^{u^{*}} \mathrm{~L}_{\mathrm{Gl}, u}\left.V\right|_{x^{+}(u)} \mathrm{d} u+\left(u^{*}\right)^{\mathrm{T}} R(x) u^{*} \leq 0
for a given positive definite RR and u∗u^{*} solution of (8) asymptotically stabilizing the equilibrium x=0x=0, then u∗u^{*} is optimal for the cost functional (3) with
l(x)=V(x)−V(F(x,u∗(x)))−(u∗(x))TR(x)u∗(x)l(x)=V(x)-V\left(F\left(x, u^{*}(x)\right)\right)-\left(u^{*}(x)\right)^{\mathrm{T}} R(x) u^{*}(x).
Proof. By construction, the function VV satisfying (15) solves the HJB equation of the form (7) with l(x)l(x) given by (16).

Under the quadraticity assumption A1, one gets the following corollary.

Corollary 3.2. Given the discrete-time dynamics (1)-(2), suppose that there exists a positive semidefinite function V(x)V(x) which satisfies A1 and the inequality
V(F0(x))−V(x)−12 LC1V(F0(x))T LC1V(F0(x))D(F0(x))=−l(x)≤0V\left(F_{0}(x)\right)-V(x)-\frac{1}{2} \frac{\mathrm{~L}_{\mathrm{C}_{1}} V\left(F_{0}(x)\right)^{\mathrm{T}} \mathrm{~L}_{\mathrm{C}_{1}} V\left(F_{0}(x)\right)}{D\left(F_{0}(x)\right)}=-l(x) \leq 0
for a given positive definite RR, then the feedback u∗u^{*} in (13), asymptotically stabilizing the equilibrium x=0x=0, is optimal for the cost functional (3).

4. Optimality and uu-average dissipativity

The characterization of the optimal solution in Theorem 3.1 with R=IdR=I_{d} suggests to investigate the links with uu-average passivity introduced by the authors in [5] to deal with generally nonlinear
dynamics. Connections between uu-average passivity and optimality are thus discussed in this section. In particular in Theorem 4.1 it is shown that inverse optimality of a certain negative output feedback us=−Y(x+(us),us)u_{s}=-Y\left(x^{+}\left(u_{s}\right), u_{s}\right) is equivalent to output average passivity with respect to Y(.,u):=LGl,uY(., u):=\mathrm{L}_{\mathrm{Gl}, u}, S, a dummy function suitably computed from the storage function SS. Again, under quadraticity in uu of Y(x+(u),u)Y\left(x^{+}(u), u\right), the optimal control solution is explicitly described in Corollary 4.1. Illustrative case studies are reported in Section 5. For, some concepts and results are recalled from [5].

Definition 4.1. Given the dynamics (1)-(2) with output Y(.,u)Y(., u) specified by the mapping Y(x,u):Rn×U→RY(x, u): R^{n} \times U \rightarrow R, one defines for any pair (x,u)(x, u) the associated uu-average output mapping denoted by Yav(x,u)Y_{a v}(x, u); i.e.
Yav(x,u):=1u∫0uY(x+(v),v)dvY_{a v}(x, u):=\frac{1}{u} \int_{0}^{u} Y\left(x^{+}(v), v\right) \mathrm{d} v
with Yav(x,0):=Y(x+(0))Y_{a v}(x, 0):=Y\left(x^{+}(0)\right)
and x+(0)=F0(x)x^{+}(0)=F_{0}(x).
We denote by Σd(Y(.,u))\Sigma_{d}(Y(., u)) the discrete-time system composed with the dynamics (1)-(2) and output mapping Y(.,u)Y(., u). The following definitions are instrumental.

Definition 4.2. Let SS be a positive semidefinite function with S(0)=0S(0)=0.
(i) AvP−Σd(Y(.,u))A_{v} P-\Sigma_{d}(Y(., u)) is said to be uu-average passive if for any pair (x,u)∈Rn×U(x, u) \in R^{n} \times U
S(x+(u))−S(x)≤∫0uY(x+(v),v)dv=uTYav(x,u)S\left(x^{+}(u)\right)-S(x) \leq \int_{0}^{u} Y\left(x^{+}(v), v\right) \mathrm{d} v=u^{T} Y_{a v}(x, u).
(ii) OFAvP(ρ)−Σd(Y(.,u))O F A_{v} P(\rho)-\Sigma_{d}(Y(., u)) is said to be output feedback uu-average passive if for any pair (x,u)∈Rn×U(x, u) \in R^{n} \times U

S(x+(u))−S(x)≤∫0uY(x+(v),v)dv+ρ(x,us)YT(x+(us),us)Y(x+(us),us)\begin{aligned} S\left(x^{+}(u)\right)-S(x) \leq & \int_{0}^{u} Y\left(x^{+}(v), v\right) \mathrm{d} v \\ & +\rho\left(x, u_{s}\right) Y^{T}\left(x^{+}\left(u_{s}\right), u_{s}\right) Y\left(x^{+}\left(u_{s}\right), u_{s}\right) \end{aligned}

for some coefficient ρ(x,us)\rho\left(x, u_{s}\right) and usu_{s} satisfying us=−Y(x+(us),us)u_{s}=-Y\left(x^{+}\left(u_{s}\right), u_{s}\right).
Definition 4.3. - zero-state detectability - ZSD - Given Σd(Y(.,u))\Sigma_{d}(Y(., u)), let Z∈Rn\mathbb{Z} \in R^{n} be the largest sub-set of states which are invariant under the unforced dynamics F0(F_{0}(.)andgivezerooutputY(F0(x),0)=) and give zero output Y\left(F_{0}(x), 0\right)= 0,Σd(Y(.,u))0, \Sigma_{d}(Y(., u)) is zero state detectable when no trajectory of the uncontrolled dynamics can stay in Z\mathbb{Z} other than those converging asymptotically to zero.

The following comments better explain these definitions.
(a) uu-average passivity has been introduced in [5] to overcome the pathology induced by the usual definition of passivity when applied to strictly causal systems. To discuss passivity for systems without direct input-output link is thus made possible.
(b) Any feedback ensuring negativity of the right hand side of the average passivity inequality (18) is an uu-average passivity based damping controller. In [5], the stabilizing properties of the feedback satisfying u=−Yav(x,u)u=-Y_{a v}(x, u) have been discussed. In particular, it is shown that assuming Lyapunov stability of the dynamics (1)-(2) ⟨V(F0(x))−V(x)≤0⟩\left\langle V\left(F_{0}(x)\right)-V(x) \leq 0\right\rangle, then uu-average passivity with respect to the output Ypb(.,u):=LGl,uV(Y^{p b}(., u):=\mathrm{L}_{\mathrm{Gl}, u} V(.)followsandthecontrollersatis−) follows and the controller satis- fying upb=−Yavpb(.,upb)u_{p b}=-Y_{a v}^{p b}\left(., u_{p b}\right) is an uu-average damping passivity based controller. Under upbu_{p b} one gets in closed loop

V(F(x,upb))−V(x)=V(F0(x))−V(x)+∫0upb LGl,v)V(x+(v))dv=V(F0(x))−V(x)−upbTupb≤0\begin{aligned} V\left(F\left(x, u_{p b}\right)\right)-V(x) & =V\left(F_{0}(x)\right)-V(x)+\int_{0}^{u_{p b}} \mathrm{~L}_{\mathrm{Gl}, v)} V\left(x^{+}(v)\right) \mathrm{d} v \\ & =V\left(F_{0}(x)\right)-V(x)-u_{p b}^{T} u_{p b} \leq 0 \end{aligned}

because by definition of upb,∫0upb LGl,v)V(x+(v))dv=−upbTupbu_{p b}, \int_{0}^{u_{p b}} \mathrm{~L}_{\mathrm{Gl}, v)} V\left(x^{+}(v)\right) \mathrm{d} v=-u_{p b}^{T} u_{p b}.

© If the output mapping in (ii) does not depend on uu, one speaks about predictive output feedback uu-average passivity because usu_{s} satisfying us=−Y(x+(us))u_{s}=-Y\left(x^{+}\left(u_{s}\right)\right) restitutes the negative one-step ahead output predicted feedback, uk=−Y(xk+1)u_{k}=-Y\left(x_{k+1}\right).
(d) The one-step ahead output predicted feedback usu_{s} can be seen as a pure output feedback vs=−Y(x)v_{s}=-Y(x) for the one-step input delayed discrete-time dynamics (1)-(2) defined on the extended state space Rn+1R^{n+1} with v(k+1)=u(k)v(k+1)=u(k).

Setting R=1R=1 into (3), the result below clarifies in terms of necessary and sufficient conditions the connections between average passivity of the dynamics and optimal stabilizing controllers. Starting from a Lyapunov function candidate SS, a novel inverse optimal control is built in the form of a negative " LGl,,u,S(\mathrm{L}_{\mathrm{Gl},, \mathrm{u},} S(.)"−feedbackun−) "- feedback un- der ZSD assumption. The function SS acts as a storage function and control Lyapunov function for the dynamics.

Theorem 4.1. Given the discrete-time dynamics (1)-(2) and a C2C^{2} positive semidefinite function S(x),us(x)S(x), u_{s}(x), solution of us=u_{s}= −Y(x+(us),us)-Y\left(x^{+}\left(u_{s}\right), u_{s}\right) with Y(.,u):=LGl,,u,SY(., u):=\mathrm{L}_{\mathrm{Gl},, u,} S is optimal stabilizing for the cost (3) if and only if the system Σd(Y)\Sigma_{d}(Y) is ZSD and OFA, P(ρ)P(\rho) with
ρ(x,us)=(12+∂Yav(x,u)∂u∣us)\rho\left(x, u_{s}\right)=\left(\left.\frac{1}{2}+\frac{\partial Y_{a v}(x, u)}{\partial u} \right|_{u_{s}}\right)
and RS(x+(us),us)+1>0\mathscr{R}_{S}\left(x^{+}\left(u_{s}\right), u_{s}\right)+1>0.
Proof. From the result in Theorem 3.1, the feedback us=u_{s}= −Y(x+(us),us)-Y\left(x^{+}\left(u_{s}\right), u_{s}\right) is optimal stabilizing for the cost (3) if it achieves asymptotic stabilization to x=0x=0 and there exists a C2C^{2} positive semidefinite function V(x)V(x) such that

To prove that these equalities are equivalent to OFA,P(ρ)O F A, P(\rho) and (20), we first note that setting S(x)=V(x)2S(x)=\frac{V(x)}{2}, these equalities rewrite

Computing now S(F(x,u))−S(x)S(F(x, u))-S(x) one gets

S(F(x,u))−S(x):=S(F0(x))−S(x)+∫0u LG(,v)S(x+(v))dv=−l(x)2−∫0usY(x+(v),v)dv−12usTus+∫0uY(x+(v),v)dv≤uYav(x,u)−usYav(x,us)−12usTus\begin{aligned} S(F(x, u))-S(x):= & S\left(F_{0}(x)\right)-S(x)+\int_{0}^{u} \mathrm{~L}_{G(, v)} S\left(x^{+}(v)\right) \mathrm{d} v \\ = & -\frac{l(x)}{2}-\int_{0}^{u_{s}} Y\left(x^{+}(v), v\right) \mathrm{d} v-\frac{1}{2} u_{s}^{T} u_{s} \\ & +\int_{0}^{u} Y\left(x^{+}(v), v\right) \mathrm{d} v \\ \leq & u Y_{a v}(x, u)-u_{s} Y_{a v}\left(x, u_{s}\right)-\frac{1}{2} u_{s}^{T} u_{s} \end{aligned}

From the definition of Yav(x,u)Y_{a v}(x, u) as ∫0uY(x+(v),v)dv:=uYav(x,u)\int_{0}^{u} Y\left(x^{+}(v), v\right) \mathrm{d} v:=\left.u Y_{a v}(x, u)\right., one gets through derivation with respect to uu the equality
Y(x+(u),u)=Yav(x,u)+u∂Yav(x,u)∂uY\left(x^{+}(u), u\right)=Y_{a v}(x, u)+u \frac{\partial Y_{a v}(x, u)}{\partial u}
and thus usYav(x,us)=usY(x+(us),us)−us∂Yav(x,u)∂u∣usu_{s} Y_{a v}\left(x, u_{s}\right)=u_{s} Y\left(x^{+}\left(u_{s}\right), u_{s}\right)-\left.u_{s} \frac{\partial Y_{a v}(x, u)}{\partial u} \right|_{u_{s}} which can be replaced into the right hand side of (21) to get
S(F(x,u))−S(x)≤uYav(x,u)+(12+∂Yav(x,u)∂u∣us)usTusS(F(x, u))-S(x) \leq u Y_{a v}(x, u)+\left.\left(\frac{1}{2}+\frac{\partial Y_{a v}(x, u)}{\partial u} \right|_{u_{s}}\right) u_{s}^{T} u_{s}
or equivalently the OFA,P(ρ)O F A, P(\rho) inequality
S(F(x,u))−S(x)S(F(x, u))-S(x)
≤uYav(x,u)+ρ(x,us)YT(x+(us),us)Y(x+(us),us)\leq u Y_{a v}(x, u)+\rho\left(x, u_{s}\right) Y^{T}\left(x^{+}\left(u_{s}\right), u_{s}\right) Y\left(x^{+}\left(u_{s}\right), u_{s}\right)
with ρ(x,us)\rho\left(x, u_{s}\right) given in (20).
Due to asymptotic stabilization under optimal control, near x=x= 0 , the solutions to xk+1=F0(xk)x_{k+1}=F_{0}\left(x_{k}\right) that satisfy LGl,,0S(F0(x))=0\mathrm{L}_{\mathrm{Gl},, 0} S\left(F_{0}(x)\right)=0 converge to 0 so that ZSD of Σd(Y(.,u))\Sigma_{d}(Y(., u)) holds.

Conversely, if the system with output Y(.,u)=LG(,u)SY(., u)=\mathrm{L}_{G(, u)} S is OFA,P(ρ)O F A, P(\rho), then usu_{s} implicitly defined by us=−LG(,us)S(x+(us))u_{s}=-\mathrm{L}_{G(, u_{s})} S\left(x^{+}\left(u_{s}\right)\right) achieves asymptotic stabilization of the equilibrium x=0x=0 and is optimal for the cost (3) with Lyapunov function V=2SV=2 S and

l(x)2=−S(x+(0))+S(x)+ρ(x,us)YT(x+(us),us)Y(x+(us),us)≥0\begin{aligned} \frac{l(x)}{2}= & -S\left(x^{+}(0)\right)+S(x) \\ & +\rho\left(x, u_{s}\right) Y^{T}\left(x^{+}\left(u_{s}\right), u_{s}\right) Y\left(x^{+}\left(u_{s}\right), u_{s}\right) \geq 0 \end{aligned}

because of inequality (23). Minimality follows from (20).
Under the quadraticity assumption A1, one gets a more constructive result.
Corollary 4.1. Given the discrete-time dynamics (1)-(2) and assume that there exist a C2C^{2} positive semidefinite function S(x)S(x) such that LGl,,u)S(x+(u))\mathrm{L}_{\mathrm{Gl},, u)} S\left(x^{+}(u)\right) is quadratic in uu, i.e.
LGl,,u)S(x+(u)):=LG1S(x+(0))+uRS(x+(0))\mathrm{L}_{\mathrm{Gl},, u)} S\left(x^{+}(u)\right):=\mathrm{L}_{G_{1}} S\left(x^{+}(0)\right)+u \mathscr{R}_{S}\left(x^{+}(0)\right).
The feedback
us=−LG1S(x+(0))1+RS(x+(0))u_{s}=-\frac{\mathrm{L}_{G_{1}} S\left(x^{+}(0)\right)}{1+\mathscr{R}_{S}\left(x^{+}(0)\right)}
is optimal stabilizing for the cost (3) if and only if the system Σd( LG(,u)S)\Sigma_{d}\left(\mathrm{~L}_{G(, u)} S\right) is ZSD and OFA,P(ρ)O F A, P(\rho) with ρ(x)=1+RS(x+(0))2\rho(x)=\frac{1+\mathscr{R}_{S}\left(x^{+}(0)\right)}{2}.

More in detail, under A1, the OFA,P(ρ)O F A, P(\rho) inequality takes the form

S(F(x,u))−S(x)≤∫0u( LG1S(x+(0))+vRS(x+(0)))dv+12( LG1S(x+(0)))T LG1S(x+(0))1+RS(x+(0))\begin{aligned} S(F(x, u))-S(x) \leq & \int_{0}^{u}\left(\mathrm{~L}_{G_{1}} S\left(x^{+}(0)\right)+v \mathscr{R}_{S}\left(x^{+}(0)\right)\right) \mathrm{d} v \\ & +\frac{1}{2} \frac{\left(\mathrm{~L}_{G_{1}} S\left(x^{+}(0)\right)\right)^{T} \mathrm{~L}_{G_{1}} S\left(x^{+}(0)\right)}{1+\mathscr{R}_{S}\left(x^{+}(0)\right)} \end{aligned}

because ρ(x)=1+RS(x+(0))2=D(x+(0))2\rho(x)=\frac{1+\mathscr{R}_{S}\left(x^{+}(0)\right)}{2}=\frac{D\left(x^{+}(0)\right)}{2} and is positive by construction. We note that the first term in the right member of the inequality (26) rewrites as uYav(x,u)u Y_{a v}(x, u) when Y(x+(u),u):=LG1S(x+(0))+Y\left(x^{+}(u), u\right):=\mathrm{L}_{G_{1}} S\left(x^{+}(0)\right)+ uRS(x+(0))u \mathscr{R}_{S}\left(x^{+}(0)\right). Optimality with Lyapunov function V=2SV=2 S and cost (3) with l(x)=−2S(x+(0))+2S(x)+(LG1S(x+(0)))T LG1S(x+(0))1+RS(x+(0))≥0l(x)=-2 S\left(x^{+}(0)\right)+2 S(x)+\frac{\left(\mathrm{L}_{G_{1}} S\left(x^{+}(0)\right)\right)^{T} \mathrm{~L}_{G_{1}} S\left(x^{+}(0)\right)}{1+\mathscr{R}_{S}\left(x^{+}(0)\right)} \geq 0 follows.
Remark 4.1. Setting u=us+vu=u_{s}+v in the inequality (21) and considering the feedback usu_{s} as a preliminary feedback transforming the drift term F0F_{0} into F~0:=F(x,us)\tilde{F}_{0}:=F\left(x, u_{s}\right), the resulting closed loop dynamics is PFA,P(ρ~)P F A, P(\tilde{\rho}) with ρ~=12\tilde{\rho}=\frac{1}{2} so verifying an excess of passivity; i.e.

S(F~(x,v))−S(x)≤vY~av(x,v)−12YT(x+(us),us)Y(x+(us),us)\begin{aligned} S(\tilde{F}(x, v))-S(x) \leq & v \tilde{Y}_{a v}(x, v) \\ & -\frac{1}{2} Y^{T}\left(x^{+}\left(u_{s}\right), u_{s}\right) Y\left(x^{+}\left(u_{s}\right), u_{s}\right) \end{aligned}

with
vY~av(x,v):=∫usus+vY(x+(w),w)dw=∫0vY~(x+(w),w)dwv \tilde{Y}_{a v}(x, v):=\int_{u_{s}}^{u_{s}+v} Y\left(x^{+}(w), w\right) \mathrm{d} w=\int_{0}^{v} \tilde{Y}\left(x^{+}(w), w\right) \mathrm{d} w.

5. Some examples and constructive cases

Some particular or constructive examples are treated below starting from the simplest one, the linear case.

5.1. The linear case

Let the representation (1)-(2) of a linear time invariant dynamics, ΣL\Sigma_{L}

x+=Ax∂x+(u)∂u=B with x+(0)=x+\begin{aligned} & x^{+}=A x \\ & \frac{\partial x^{+}(u)}{\partial u}=B \quad \text { with } x^{+}(0)=x^{+} \end{aligned}

with AA and BB matrices of appropriate dimensions. Considering as usual the quadratic cost J=∑kxkTQxk+ukTRuk,R>0J=\sum_{k} x_{k}^{\mathrm{T}} Q x_{k}+u_{k}^{\mathrm{T}} R u_{k}, R>0 and Q≥0Q \geq 0, the optimal control takes the form
u∗=−BTPAxR+BTPBu^{*}=-\frac{B^{\mathrm{T}} P A x}{R+B^{\mathrm{T}} P B}
with optimal value function V(x)=xTPxV(x)=x^{\mathrm{T}} P x and matrix PP solution of the Riccati equation P=Q+AT(P−PB(R+BTPB)−1BTP)AP=Q+A^{\mathrm{T}}\left(P-P B\left(R+B^{\mathrm{T}} P B\right)^{-1} B^{\mathrm{T}} P\right) A.

Accordingly the result in Theorem 4.1 gives.
Proposition 5.1. Given ΣL\Sigma_{L} and a(n×n)a(n \times n) symmetric matrix S≥0S \geq 0. Setting C=2BTSC=2 B^{\mathrm{T}} S, the negative predicted output feedback uk=u_{k}= −Cxk+1=−CA1+CBxk-C x_{k+1}=-\frac{C A}{1+C B} x_{k} is optimal stabilizing if and only if the system ΣL(C)\Sigma_{L}(C) is detectable and POFA⁡vP(ρ)\operatorname{POFA}_{v} P(\rho) with ρ=1+CB2\rho=\frac{1+C B}{2} and storage function xTSxx^{\mathrm{T}} S x.

We note that in this case ρ=1+CB2\rho=\frac{1+C B}{2} since by definition Yav(x,u)=CAx+uCB2Y_{a v}(x, u)=C A x+u^{\frac{C B}{2}} and the condition ρ>0\rho>0 is satisfied with C=2BTSC=2 B^{\mathrm{T}} S. The cost QQ is given by
Q=−2ATSA+2S+ATCTCA1+CBQ=-2 A^{\mathrm{T}} S A+2 S+\frac{A^{\mathrm{T}} C^{\mathrm{T}} C A}{1+C B}
Moreover, the POFA⁡vP(ρ)\operatorname{POFA}_{v} P(\rho) condition (19) rewrites as
(xTAT+uTBT)S(Ax+Bu)−xTSx≤∫0uCx+(v)dv+xTATCTCAx2(1+CB)\left(x^{\mathrm{T}} A^{\mathrm{T}}+u^{\mathrm{T}} B^{\mathrm{T}}\right) S(A x+B u)-x^{\mathrm{T}} S x \leq \int_{0}^{u} C x^{+}(v) \mathrm{d} v+\frac{x^{\mathrm{T}} A^{\mathrm{T}} C^{\mathrm{T}} C A x}{2(1+C B)}
where the positive term xTATCTCB2(1+CB)\frac{x^{\mathrm{T}} A^{\mathrm{T}} C^{\mathrm{T}} C B}{2(1+C B)} characterizes the shortage of average passivity with respect to the output map CC.

5.2. The case G(x,u)=G(x)G(x, u)=G(x)

Consider

x+=F0(x)x^{+}=F_{0}(x)
∂x+(u)∂u=G(x+(u))\frac{\partial x^{+}(u)}{\partial u}=G\left(x^{+}(u)\right) \quad with x+(0)=x+x^{+}(0)=x^{+}
obtained from (1) and (2) with G(x,u)=G(x)G(x, u)=G(x). This is a particular class of discrete-time dynamics, their input-to-state and input-to-output behaviors admit the exponential representations x+(u)=eaG(F0(x))x^{+}(u)=e^{a G}\left(F_{0}(x)\right) and Y(x+(u))=eaGY(F0(x))Y\left(x^{+}(u)\right)=e^{a G} Y\left(F_{0}(x)\right) and, as shown in [20], they exhibit structural and geometric properties somehow similar to input-affine continuous-time dynamics. This similarity is reinforced by the result obtained in the present context. As a matter of fact, the result in Theorem 4.1 can be restated as follows:

Proposition 5.2. Given the discrete-time dynamics (28)-(29) and a C+C^{+}C+positive semidefinite function S(x)S(x), setting Y(⋅)=LGS(⋅)Y(\cdot)=\mathrm{L}_{G} S(\cdot), the predicted output feedback
uk=−LGS(x+(uk))=Yk+1u_{k}=-\mathrm{L}_{G} S\left(x^{+}\left(u_{k}\right)\right)=Y_{k+1}
is optimal stabilizing for the cost (3) if and only if the system Σd(Y)\Sigma_{d}(Y) is ZSD and POFA⁡vP(ρ)\operatorname{POFA}_{v} P(\rho) with storage function S(x)S(x) and ρ(x,uk)=\rho\left(x, u_{k}\right)= 12+ddu∣ukYav(x,u)\left.\frac{1}{2}+\frac{\mathrm{d}}{\mathrm{d} u}\right|_{u_{k}} Y_{a v}(x, u); i.e. it satisfies
S(x+(u))−S(x)≤uYav(x,u)+ρ(x,uk)Yk+12S\left(x^{+}(u)\right)-S(x) \leq u Y_{a v}(x, u)+\rho\left(x, u_{k}\right) Y_{k+1}^{2}.

More in detail, in such a case:

uk=−Y(x+(uk))=−LGS(x+(uk))=−eukG∘ LGS(F0(xk))u_{k}=-Y\left(x^{+}\left(u_{k}\right)\right)=-\mathrm{L}_{G} S\left(x^{+}\left(u_{k}\right)\right)=-e^{u_{k} G} \circ \mathrm{~L}_{G} S\left(F_{0}\left(x_{k}\right)\right)
with Y=LGSY=\mathrm{L}_{G} S and x+(uk)=eukG(F0(xk))x^{+}\left(u_{k}\right)=e^{u_{k} G}\left(F_{0}\left(x_{k}\right)\right);

Remark 5.1. In particular when in (29), G(x)=BG(x)=B, a constant matrix, one recovers a subclass xk+1=F0(xk)+Bukx_{k+1}=F_{0}\left(x_{k}\right)+B u_{k} of input-affine dynamics studied below for which passivity like conditions can be formulated in terms of LMI. It is of practical interest to note that many different nonlinear dynamics admit such representations as mechanical systems (spacecraft, ball and bean, robots with flexible joints) or electrical machines (induction motors, synchronous generators) for which constructive solutions exist.

5.3. The input-affine case

Input-affine dynamics of the form
xk+1=A(xk)xk+B(xk)ukx_{k+1}=A\left(x_{k}\right) x_{k}+B\left(x_{k}\right) u_{k}
are currently discussed in the nonlinear discrete-time literature [9,15,21]. In the present context, the resulting property of the associated control vector field G(.,u)G(., u) is quite involved. However, when considering a quadratic Lyapunov function candidate V(x)=V(x)= xTPxx^{\mathrm{T}} P x, assumption A1 is satisfied so getting
G(x+(u),u):=B(x)G\left(x^{+}(u), u\right):=B(x)
LG(.,u)V(x+(u)):=2BT(x)Px+(u)\mathrm{L}_{G(., u)} V\left(x^{+}(u)\right):=2 B^{\mathrm{T}}(x) P x^{+}(u)
RV(x+(u),u):=2BT(x)PB(x)\mathscr{R}_{V}\left(x^{+}(u), u\right):=2 B^{\mathrm{T}}(x) P B(x)
Corollaries 3.2 and 4.1 hold true and are constructive for the optimal stabilizing feedbacks so reformulating results in [15] in terms of POFA⁡VP(ρ)\operatorname{POFA}_{V} P(\rho) and average passivity. As an example, consider the input-affine dynamics proposed in [9].
x1(k+1)=x2(k)x_{1}(k+1)=x_{2}(k)
x2(k+1)=−x1(k)+b(x1(k),x2(k))u(k)x_{2}(k+1)=-x_{1}(k)+b\left(x_{1}(k), x_{2}(k)\right) u(k)
where b(x1,x2)b\left(x_{1}, x_{2}\right) is an arbitrary smooth function from R2R^{2} to RR with b(0,0)=0b(0,0)=0 and such that b(x1,(+−)x2)=0b\left(x_{1},(+-) x_{2}\right)=0 implies x1=0x_{1}=0. Setting V(x)=12(x12+x22)V(x)=\frac{1}{2}\left(x_{1}^{2}+x_{2}^{2}\right), one verifies that V(Ax)=V(x)V(A x)=V(x) and thus average passivity of the dynamics with output LG(.,u)V(x+(u))=\mathrm{L}_{G(., u)} V\left(x^{+}(u)\right)= −x1b(x1,x2)+b2(x1,x2)u-x_{1} b\left(x_{1}, x_{2}\right)+b^{2}\left(x_{1}, x_{2}\right) u. More precisely, the system is uu-average lossless

V(x+(u))−V(x)=∫0u LG(.,v)V(x+(v))dv=u( LG(.,v)V)av(x,u)=u(−x1b(x1,x2)+u2b2(x1,x2))\begin{aligned} V\left(x^{+}(u)\right)-V(x) & =\int_{0}^{u} \mathrm{~L}_{G(., v)} V\left(x^{+}(v)\right) \mathrm{d} v \\ & =u\left(\mathrm{~L}_{G(., v)} V\right)_{a v}(x, u) \\ & =u\left(-x_{1} b\left(x_{1}, x_{2}\right)+\frac{u}{2} b^{2}\left(x_{1}, x_{2}\right)\right) \end{aligned}

It follows that the negative average output feedback
upb=−(LG(.,upb)V)av(x,upb)=x1b(x1,x2)1+b2(x1,x2)2u_{p b}=-\left(\mathrm{L}_{G(., u_{p b})} V\right)_{a v}\left(x, u_{p b}\right)=\frac{x_{1} b\left(x_{1}, x_{2}\right)}{1+\frac{b^{2}\left(x_{1}, x_{2}\right)}{2}}
is asymptotically stabilizing as shown in [9] because ZSD clearly holds from the properties of b(x1,x2)b\left(x_{1}, x_{2}\right). On the other hand, the predicted output feedback proposed in this paper, namely the feedback
u=−LG(.,u)V(x+(u))=x1b(x1,x2)1+b2(x1,x2)u=-\mathrm{L}_{G(., u)} V\left(x^{+}(u)\right)=\frac{x_{1} b\left(x_{1}, x_{2}\right)}{1+b^{2}\left(x_{1}, x_{2}\right)}

is asymptotically optimal stabilizing for the cost (3) with l(x)=l(x)= x12b2(x1,x2)1+b2(x1,x2)\frac{x_{1}^{2} b^{2}\left(x_{1}, x_{2}\right)}{1+b^{2}\left(x_{1}, x_{2}\right)} and Lyapunov function 2V(x)2 V(x) because the system is ZSD and POFA⁡VP(ρ)\operatorname{POFA}_{V} P(\rho) with ρ=12(1+b2(x1,x2))\rho=\frac{1}{2}\left(1+b^{2}\left(x_{1}, x_{2}\right)\right); i.e.
2V(x+(u))−2V(x)≤∫0uLG(.,v)V(x+(v))dv+12x12b2(x1,x2)(1+b2(x1,x2))2 V\left(x^{+}(\mathrm{u})\right)-2 V(x) \leq \int_{0}^{\mathrm{u}} \mathrm{L}_{\mathrm{G}(., v)} V\left(x^{+}(v)\right) \mathrm{d} v+\frac{1}{2} \frac{x_{1}^{2} b^{2}\left(x_{1}, x_{2}\right)}{\left(1+b^{2}\left(x_{1}, x_{2}\right)\right)}.
As noted in [9], this example includes a family of planar discretetime dynamics for which the asymptotic stabilization cannot be solved by either the linear approximation or the center manifold approach for maps.

6. Conclusions

Adopting the differential/difference representation of nonlinear discrete-time dynamics, the optimality conditions of stabilizing controllers have been discussed relative to passivity properties. The proposed framework makes it possible to deal with general nonlinear dynamics so extending results formalized in the literature under quadraticity assumptions on the nonlinearities. The computation of the control law, addressed in particular cases in this work, remains a difficult task. For dynamics issued from the sampling of input-affine continuous-time ones it becomes possible to solve the implicit equality (8) through executable algorithms. The computational feasibility of sampled-data controllers with respect to purely discrete-time solutions has been already discussed in [14] in the context of passivity based controllers and deserves interesting peculiarities which will be the object of further investigations.

References

[1] D. Hill, P.J. Moylan, The stability of nonlinear dissipative systems, IEEE Trans. Automat. Control 21 (1976) 708-711.
[2] R.A. Freeman, P.V. Kokotovic, Robust Nonlinear Control Design: State-Space and Lyapunov Techniques, Birkhauser, Boston, MA, 1996.
[3] R. Sepuchre, M. Jankovic, P. Kokotovic, Constructive Nonlinear Control, Springer, New York, 1997.
[4] R. Ortega, E. Garcia-Canseco, Interconnection and damping assignment passivity-based control: a survey, Eur. J. Control 10 (2004) 432-450.
[5] S. Monaco, D. Normand-Cyrot, Nonlinear average passivity and stabilizing controllers in discrete-time, Systems Control Lett. 60 (2011) 431-439.
[6] S. Monaco, D. Normand-Cyrot, Nonlinear discrete-time representations, a new paradigm, in: D. Normand-Cyrot (Ed.), Perspectives in Control, a Tribute to I.D. Landau, Springer Londres, 1998, pp. 191-205.
[7] C.I. Byrnes, W. Lin, Losslessness, feedback equivalence and the global stabilization of discrete-time nonlinear systems, IEEE Trans. Automat. Control 39 (1994) 83-98.
[8] W. Lin, C.I. Byrnes, KYP lemma, state feedback and dynamic output feedback in discrete-time bilinear systems, Systems Control Lett. 23 (1994) 127-136.
[9] W. Lin, C.I. Byrnes, Passivity and absolute stabilization of a class of discretetime nonlinear systems, Automatica 31 (1995) 263-267.
[10] H. Guillard, S. Monaco, D. Normand-Cyrot, An approach to nonlinear discrete time H-infinity control, in: Proc. 32nd IEEE-CDC San Antonio, 1993, pp. 178-183.
[11] D. Laila, A. Astolfi, On the construction of discrete-time model for portcontrolled Hamiltonian systems with applications, Systems Control Lett. 55 (2006) 673-680.
[12] D. Nesic, A.R. Teel, P. Kokotovic, Sufficient conditions for stabilization of sampled-data nonlinear systems via discrete-time approximations, Systems Control Lett. 38 (1999) 259-270.
[13] D. Nesic, A.R. Teel, A framework for stabilization of nonlinear sampleddata systems based on their approximate discrete-time models, IEEE Trans. Automat. Control 49 (2004) 1034-1103.
[14] S. Monaco, D. Normand-Cyrot, F. Tiefensee, Sampled-data stabilizing feedback; a PBC approach, IEEE Trans. Automat. Control 56 (2011) 907-912.
[15] F. Ornelas, E.N. Sanchez, A.G. Loukianov, Inverse optimal control for discretetime nonlinear systems via passivation, Optim. Control Appl. Meth. 35 (2014) 110-126.
[16] S. Monaco, D. Normand-Cyrot, Nonlinear optimal stabilizing control in discrete-time, in: Proc. ACC’12, Montréal, 2012, pp. 208-213.
[17] E.M. Navarro-López, H. Sira-Ramirez, E. Fossas-Colet, Dissipativity and feedback dissipativity of general nonlinear discrete-time systems, in: Dissipativity of Dynamical Systems, Eur. J. Control 8 (2002) 265-274 (special issue).
[18] T. Rasar, G.J. Oöder, Dynamic Noncooperative Game Theory, Academic Press, London, New York, 1995.
[19] F. Mazenc, H. Nijmeijer, Forwarding in discrete-time nonlinear systems, Int. J. Control 71 (1998) 823-835.
[20] S. Monaco, D. Normand-Cyrot, Geometric properties of a class of nonlinear discrete-time dynamics dynamics, Invited Paper, Proc. European Control Conference, ECC-97, Bruxelles (1997).
[21] E.N. Sanchez, F. Ornelas-Tellez, in: Taylor, Francis (Eds.), Discrete-time Inverse Optimal Control for Nonlinear Systems, CRC Press, 2014.
[22] S. Monaco, D. Normand-Cyrot, C. Califano, From chronological calculus to exponential representations of continuous and discrete-time dynamics: a Lie algebraic approach, IEEE Trans. Automat. Control 52 (2007) 2227-2241.