A New Framework for Bounding Reachability Probabilities of Continuous-time Stochastic Systems1footnote 11footnote 1To appear in Nonlinear Analysis: Hybrid Systems (original) (raw)

Abstract

This manuscript presents an innovative framework for constructing barrier functions to bound reachability probabilities for continuous-time stochastic systems described by stochastic differential equations (SDEs). The reachability probabilities considered in this paper encompass two aspects: the probability of reaching a set of specified states within a predefined finite time horizon, and the probability of reaching a set of specified states at a particular time instant. The barrier functions presented in this manuscript are developed either by relaxing a parabolic partial differential equation that characterizes the exact reachability probability or by applying the Grönwall’s inequality. In comparison to the prevailing construction method, which relies on Doob’s non-negative supermartingale inequality (or Ville’s inequality), the proposed barrier functions provide stronger alternatives, complement existing methods, or fill gaps.

keywords:

Stochastic Differential Equations, Reachability Probabilities

\affiliation

organization=Key Laboratory of System Software (Chinese Academy of Sciences) and State Key Lab. of Computer Science, Institute of Software Chinese Academy of Sciences,city=Beijing, postcode=100190, country=China

1 Introduction

Stochastic phenomena are commonly observed in both natural and artificial systems, spanning multiple disciplines such as biology and robotics. To accurately model these systems, sophisticated approaches are needed due to their inherent randomness [franzle2008stochastic]. Stochastic differential equations (SDEs) provide a powerful tool by integrating deterministic dynamics with stochastic processes, offering a comprehensive framework for comprehending the behavior of these systems [kloeden1992stochastic]. They have been widely applied, such as in models of disturbances in engineered systems like wind forces [wang2015long] and pedestrian motion [hoogendoorn2004pedestrian].

The reachability probability is a critical quantitative measure within the context of SDEs [lavaei2022automated]. It provides valuable insights into the likelihood of a system, governed by an SDE, reaching a set of specified states within a predetermined (in)finite time frame (referred to as reachability probability within (in)finite time horizons) or at a particular time instant (referred to as reachability probability at specific time instants). This concept plays a pivotal role in understanding the probabilistic evolution of systems under stochastic influences, enabling informed analysis and decision-making in various fields. Computing reachability probabilities typically involves solving Hamilton-Jacobi-Bellman equations [koutsoukos2008computational, bujorianu2009stochastic, esfahani2016stochastic]. However, obtaining analytical solutions is often infeasible, necessitating the use of numerical approximations. As a result, obtaining both upper and lower bounds of reachability probabilities becomes impractical. [nilsson2020lyapunov] gave comparison results for SDEs that via a Lyapunov-like function allow reachability probabilities within finite time horizons to be upper-bounded by an exit probability of a one-dimensional Ornstein-Uhlenbeck process, but the bounds are not in closed form. Inspired by Lyapunov functions for stability analysis, determining upper and lower bounds of reachability probabilities within infinite and finite time horizons has been simplified by finding barrier certificates. The primary focus of safety verification studies lies in calculating upper bounds, where the objective is to estimate the maximum likelihood of reaching a specified unsafe set. Conversely, when the goal is to assess the probability of reaching a target set, the emphasis is on obtaining a lower bound of this probability. With the development of polynomial optimization, specifically sum-of-squares polynomial optimization [parrilo2003semidefinite], barrier certificates have emerged as a powerful tool for certifying upper bounds of reachability probabilities. When the system of interest is polynomial, the problem of finding barrier certificates can be addressed through convex optimization. Barrier certificates for SDEs were initially introduced in [prajna2004stochastic, prajna2007framework] for infinite time safety verification, upper-bounding the probability that a system will eventually reach an unsafe region based on a non-negative barrier function. They build upon the known Doob’s non-negative supermartingale inequality [doob1939jean], which requires the expectation of the non-negative barrier certificate to decrease along the system dynamics. Later, inspired by results in [kushner1967stochastic] and the Doob’s non-negative supermartingale inequality, [steinhardt2012finite] extended barrier certificates to safety verification over finite time horizons and proposed c-martingales, which allow the expected value of the barrier function to increase over time. This approach provides upper bounds for the reachability probability of a system entering an unsafe region within finite time horizons. The c-martingales were further enhanced in [santoyo2021barrier] for safety verification over finite time horizons by imposing a state-dependent bound on the expected value of the barrier certificate. Recently, a controlled version was presented in [wang2021safety]. Meanwhile, [feng2020unbounded] proposed a time-varying barrier function to upper bound the reachability probability within finite time horizons, utilizing Doob’s non-negative supermartingale inequality. While there has been considerable research focusing on providing upper bounds for the reachability probability within (in)finite time horizons, the practice of lower-bounding this probability has received considerably less attention. A novel equation, which can characterize the exact reachability probability within the infinite time horizon, was proposed in [xue2021reach, Xue2023]. By relaxing this equation, barrier-like conditions can be obtained to both lower-bound and upper-bound the reachability probability within the infinite time horizon. Recently, this approach has been extended to lower-bound the reachability probability within finite time horizons in [xue2023reachability, xue2023safe]. All of the aforementioned works study the bounding problem of reachability probabilities within (in)finite time horizons. However, to the best of our knowledge, there is no work in the framework of barrier functions investigating the problem of bounding reachability probabilities at specific time instants. This is an important problem because it addresses scenarios where system performance is critical at precise moments, not just within an interval. For instance, in multi-agent systems, agents may need to synchronize their states (e.g., achieve a rendezvous) at an exact time. Similarly, in digital control, the state is typically sampled and controlled at discrete instants, making its properties at those specific times paramount for stability and performance. Providing formal bounds for these exact-time reachability probabilities thus enables verification of stricter temporal specifications and enhances the applicability of barrier certificate methods to time-critical stochastic systems.

This paper explores the issue of lower- and upper-bounding reachability probabilities within finite time horizons and at specific time instants in stochastic systems modeled by SDEs. To tackle these problems, we propose time-dependent and time-independent barrier functions that provide lower and upper bounds for these reachability probabilities. The development of these barrier certificates is influenced by our previous work [Xue2023, xue2023safe], which introduces an alternative method that does not rely on the commonly used Doob’s non-negative supermartingale inequality. Leveraging the occupation measure, the construction of the barrier certificates is achieved through either relaxation of a second-order partial differential equation or utilization of the Grönwall inequality. These barrier certificates are either more powerful compared to those found in previous works, complement the existing ones, or fill a gap. They will facilitate the gain of tight bounds on reachability probabilities within finite time horizons and at specific time instants.

The main contributions of this work are summarized below.

1. 1. Novel Methodological Foundation: The framework provides an alternative to the prevailing method that relies on Doob’s nonnegative supermartingale inequality (as used in works like [5] and [18]). Instead, it constructs barrier functions by relaxing a parabolic partial differential equation (PDE) that characterizes the exact reachability probability or by applying Grönwall’s inequality.
2. 1. Comprehensive Bounding: The work provides methods for both upper- and lower-bounding two types of reachability probabilities:
   2. (a)
      The probability of reaching a set within a finite time horizon.
   3. (b)
      The probability of being in a set at a specific time instant (a problem not previously addressed within the barrier function framework for SDEs).
3. 1. Alternative Formulation and Filled Gaps: A key difference from supermartingale-based approaches is that our conditions do not inherently require the barrier function to be non-negative everywhere for all time. This alternative formulation:
   2. (a)
      Complements existing supermartingale-based methods (e.g., [feng2020unbounded], [santoyo2021barrier]) and can, in practice, lead to the discovery of barrier functions that might not satisfy stricter non-negativity constraints.
   3. (b)
      Fills a significant gap by providing a practical approach for lower-bounding reachability probabilities, which has received considerably less attention in the literature.
   4. (c)
      Extends the reachability analysis to the novel case of probabilities at a specific time instant.

This paper is structured as follows. In Section 2, we introduce SDEs and the problems related to bounding reachability probabilities within finite time horizons and at specific time instants. In Section 3, we present our time-dependent and time-independent barrier functions for lower- and upper-bounding reachability probabilities within finite time horizons. Then, in Section 4, we present our barrier functions for lower- and upper-bounding reachability probabilities at specific time instants. Finally, in Section 6, we conclude the paper.

Some basic notions are used throughout this paper: ℝ\mathbb{R} and ℝ≥0\mathbb{R}_{\geq 0} stand for the set of real numbers and non-negative real numbers, respectively;ℝn\mathbb{R}^{n} and ℝn×m\mathbb{R}^{n\times m} denote the space of all nn-dimensional vectors and n×mn\times m real matrices, respectively; for a set 𝒜\mathcal{A}, the sets 𝒜∘\mathcal{A}^{\circ}, 𝒜¯\overline{\mathcal{A}}, and ∂𝒜\partial\mathcal{A} denote the interior, the closure, and the boundary of the set 𝒜\mathcal{A}, respectively; ∧\wedge denotes the logical operation of conjunction.

2 Preliminaries

This section introduces SDEs and the reachability probabilities bounding problem of interest.

Consider the continuous-time stochastic system,

where 𝒃​(⋅):ℝn→ℝn\bm{b}(\,\cdot\,)\colon\mathbb{R}^{n}\rightarrow\mathbb{R}^{n} and 𝝈​(⋅):ℝn→ℝn×k\bm{\sigma}(\,\cdot\,)\colon\mathbb{R}^{n}\rightarrow\mathbb{R}^{n\times k} are locally Lipschitz continuous function; 𝑾​(⋅,⋅):ℝ×Ω→ℝk\bm{W}(\cdot,\cdot)\colon\mathbb{R}\times\Omega\rightarrow\mathbb{R}^{k} is a standard kk-dimensional Wiener process, and Ω\Omega, equipped with the probability measure ℙ\mathbb{P}, is the sample space 𝒘\bm{w} belongs to. The expectation with respect to ℙ\mathbb{P} is denoted by 𝔼​[⋅]\mathbb{E}[\,\cdot\,].

Given an initial state 𝒙0\bm{x}_{0}, the SDE (1) has a unique (maximal local) strong solution over a time interval [0,T𝒙0​(𝒘))[0,T^{\bm{x}_{0}}(\bm{w})), where T𝒙0​(𝒘)T^{\bm{x}_{0}}(\bm{w}) is a positive real value or infinity. This solution is denoted as 𝑿𝒙0𝒘​(⋅):[0,T𝒙0​(𝒘))→ℝn\bm{X}_{\bm{x}_{0}}^{\bm{w}}(\,\cdot\,)\colon[0,T^{\bm{x}_{0}}(\bm{w}))\rightarrow\mathbb{R}^{n}, which satisfies the stochastic integral equation,

Given a function v​(t,𝒙)v(t,\bm{x}) that is twice continuously differentiable over 𝒙\bm{x} and continuously differentiable over tt, the infinitesimal generator underlying system (1) on the function v​(t,𝒙)v(t,\bm{x}), which represents the limit of the expected value of v​(t,𝑿𝒙0𝒘​(t))v(t,\bm{X}_{\bm{x}_{0}}^{\bm{w}}(t)) as tt approaches 0, is presented in Definition 1.

Definition 1 ([oksendal2013stochastic]).

Given system (1), the infinitesimal generator is the operator ℒ\mathcal{L}, which is defined to act on suitable functions v​(⋅,⋅):ℝ×ℝn→ℝv(\cdot,\cdot):\mathbb{R}\times\mathbb{R}^{n}\rightarrow\mathbb{R} by

The domain of ℒ\mathcal{L} is by definition the set of functions vv for which the limit (2) exists for all 𝐱∈ℝn\bm{x}\in\mathbb{R}^{n} and t∈ℝt\in\mathbb{R}.

The following proposition presents the infinitesimal generator ℒ\mathcal{L}.

Proposition 1 ([oksendal2013stochastic]).

Given system (1), the infinitesimal generator ℒ\mathcal{L} on a test function v​(t,𝐱)v(t,\bm{x}) is

where ∂v∂t\frac{\partial v}{\partial t} and ∂v∂𝐱\frac{\partial v}{\partial\bm{x}} represent the gradient of the test function v​(t,𝐱)v(t,\bm{x}) with respect to tt and 𝐱\bm{x}, respectively, ∂2v∂𝐱2\frac{\partial^{2}v}{\partial\bm{x}^{2}} represents the second-order partial derivative of the test function v​(t,𝐱)v(t,\bm{x}) with respect to 𝐱\bm{x}, and tr​(⋅)\textbf{\rm tr}(\,\cdot\,) denotes the trace of a matrix.

The domain of ℒ\mathcal{L} contains the set of functions vv of compact support, which are twice continuously differentiable over 𝐱\bm{x} and continuously differentiable over tt.

Given a state constrained set 𝒳⊆ℝn\mathcal{X}\subseteq\mathbb{R}^{n} that is open and bounded, and a subset 𝒳s⊆𝒳\mathcal{X}_{s}\subseteq\mathcal{X} that is closed, the reachability probability within a time horizon [0,T][0,T] with 0<T<∞0<T<\infty is the probability of system (1), starting from an initial state 𝒙0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}, reaches the set 𝒳s\mathcal{X}_{s} within the time horizon [0,T][0,T] while remaining within the state constrained set 𝒳\mathcal{X} until the first occurrence of hitting the set 𝒳s\mathcal{X}_{s}. It is formulated in Definition 2.

Definition 2 (Reachability Probability I).

Given a time horizon [0,T][0,T] with 0<T<∞0<T<\infty and an initial state 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}, the reachability probability ℙ𝐱0[0,T]\mathbb{P}_{\bm{x}_{0}}^{[0,T]} within the time horizon [0,T][0,T] for system (1) starting from an initial state 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s} is defined below:

| ℙ𝒙0[0,T]:=ℙ​({𝒘∈Ω|∃t∈[0,T].𝑿𝒙0𝒘​(t)∈𝒳s∧∀τ∈[0,t).𝑿𝒙0𝒘​(τ)∈𝒳}).\mathbb{P}_{\bm{x}_{0}}^{[0,T]}:=\mathbb{P}\left(\left\{\bm{w}\in\Omega\,\middle|\,\begin{aligned} \exists t\in[0,T].\bm{X}_{\bm{x}_{0}}^{\bm{w}}(t)\in\mathcal{X}_{s}\wedge\forall\tau\in[0,t).\bm{X}_{\bm{x}_{0}}^{\bm{w}}(\tau)\in\mathcal{X}\end{aligned}\right\}\right). | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |

For continuous-time systems with continuous sample paths and for any initial state 𝒙0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}, the event formulation in Definition 2,

is equivalent to the formulation ∃t∈[0,T]:𝑿𝒙0𝒘​(t)∈𝒳s∧∀τ∈[0,t):𝑿𝒙0𝒘​(τ)∈𝒳∖𝒳s\exists t\in[0,T]:\bm{X}_{\bm{x}_{0}}^{\bm{w}}(t)\in\mathcal{X}_{s}\;\wedge\;\forall\tau\in[0,t):\bm{X}_{\bm{x}_{0}}^{\bm{w}}(\tau)\in\mathcal{X}\setminus\mathcal{X}_{s}. This equivalence holds because path continuity ensures the existence of a first hitting time tt for the set 𝒳s\mathcal{X}_{s}. Consequently, the condition ∀τ∈[0,t)​𝑿𝒙0𝒘​(τ)∈𝒳\forall\tau\in[0,t)\,\bm{X}_{\bm{x}_{0}}^{\bm{w}}(\tau)\in\mathcal{X} implicitly requires the trajectory to remain in 𝒳∖𝒳s\mathcal{X}\setminus\mathcal{X}_{s} for all times before tt.

Given a state constrained set 𝒳⊆ℝn\mathcal{X}\subseteq\mathbb{R}^{n} that is open and bounded, and a subset 𝒳s⊆𝒳\mathcal{X}_{s}\subseteq\mathcal{X} that is closed, the reachability probability at a time instant TT with 0<T<∞0<T<\infty is the probability of system (1), starting from an initial state 𝒙0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}, reaches the set 𝒳s\mathcal{X}_{s} at the time instant TT while remaining within the state constrained set 𝒳\mathcal{X} before the time TT. It is formulated in Definition 3.

Definition 3 (Reachability Probability II).

Given a time horizon [0,T][0,T] with 0<T<∞0<T<\infty and an initial state 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}, the reachability probability ℙ𝐱0T\mathbb{P}_{\bm{x}_{0}}^{T} at the time instant T>0T>0 for system (1) starting from an initial state 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s} is defined below:

| ℙ𝒙0T:=ℙ​({𝒘∈Ω|𝑿𝒙0𝒘​(T)∈𝒳s∧∀τ∈[0,T).𝑿𝒙0𝒘​(τ)∈𝒳}).\mathbb{P}_{\bm{x}_{0}}^{T}:=\mathbb{P}\big(\left\{\bm{w}\in\Omega\,\middle|\,\begin{aligned} \bm{X}_{\bm{x}_{0}}^{\bm{w}}(T)\in\mathcal{X}_{s}\wedge\forall\tau\in[0,T).\bm{X}_{\bm{x}_{0}}^{\bm{w}}(\tau)\in\mathcal{X}\end{aligned}\right\}\big). | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |

In this paper, we address the challenge of computing the exact reachability probabilities in Definition 2 and 3, which is often infeasible for nonlinear systems. Instead, we resort to characterizing their lower and upper bounds, i.e., we will characterize δi,1∈[0,1]\delta_{i,1}\in[0,1] and δi,2∈[0,1]\delta_{i,2}\in[0,1], i=1,2i=1,2, such that

and

3 Bounding Reachability Probabilities I

This section introduces our barrier functions for upper- and lower-bounding the reachability probability ℙ𝒙0[0,T]\mathbb{P}_{\bm{x}_{0}}^{[0,T]} in Definition 2. They are respectively formulated in Subsection 3.1 and 3.2.

The construction of these barrier functions lies on an auxiliary stochastic process {𝑿^𝒙0𝒘​(t),t∈ℝ≥0}\{\widehat{\bm{X}}_{\bm{x}_{0}}^{\bm{w}}(t),t\in\mathbb{R}_{\geq 0}\} for 𝒙0∈𝒳∖𝒳s¯\bm{x}_{0}\in\overline{\mathcal{X}\setminus\mathcal{X}_{s}} that is a stopped process corresponding to {𝑿𝒙0𝒘​(t),t∈[0,T𝒙0​(𝒘))}\{\bm{X}_{\bm{x}_{0}}^{\bm{w}}(t),t\in[0,T^{\bm{x}_{0}}(\bm{w}))\} and the set 𝒳∖𝒳s¯\overline{\mathcal{X}\setminus\mathcal{X}_{s}}, i.e.,

where τ𝒙0​(𝒘)=inf{t∣𝑿𝒙0𝒘​(t)∈∂𝒳∪∂𝒳s}\tau^{\bm{x}_{0}}(\bm{w})=\inf\{\,t\mid\bm{X}_{\bm{x}_{0}}^{\bm{w}}(t)\in\partial\mathcal{X}\cup\partial\mathcal{X}_{s}\,\} is the first time of exit of 𝑿𝒙0𝒘​(t)\bm{X}_{\bm{x}_{0}}^{\bm{w}}(t) from the set 𝒳∖𝒳s\mathcal{X}\setminus\mathcal{X}_{s}. It is worth remarking that the first time of exit τ𝒙0​(𝒘)\tau^{\bm{x}_{0}}(\bm{w}) is well-defined and τ𝒙0​(𝒘)≤T𝒙0​(𝒘)\tau^{\bm{x}_{0}}(\bm{w})\leq T^{\bm{x}_{0}}(\bm{w}). This holds because the state constrained set 𝒳\mathcal{X} is bounded. Consequently, if the path 𝑿𝒙0𝒘​(t)\bm{X}_{\bm{x}_{0}}^{\bm{w}}(t) has a finite explosion time T𝒙0​(𝒘)<∞T^{\bm{x}_{0}}(\bm{w})<\infty (i.e., it escapes to infinity), it must first exit the bounded domain 𝒳∖𝒳s\mathcal{X}\setminus\mathcal{X}_{s}. Exiting this domain necessitates touching its boundary, ∂𝒳∪∂𝒳s\partial\mathcal{X}\cup\partial\mathcal{X}_{s}, at a time prior to the explosion.The stopped process 𝑿^𝒙0𝒘​(t)\widehat{\bm{X}}_{\bm{x}_{0}}^{\bm{w}}(t) inherits the right continuity and strong Markovian property of 𝑿𝒙0𝒘​(t)\bm{X}_{\bm{x}_{0}}^{\bm{w}}(t). Moreover, the infinitesimal generator ℒ^\widehat{\mathcal{L}} on a test function v​(t,𝒙)v(t,\bm{x}) corresponding to 𝑿^𝒙0𝒘​(t)\widehat{\bm{X}}_{\bm{x}_{0}}^{\bm{w}}(t) is identical to the one corresponding to 𝑿𝒙0𝒘​(t)\bm{X}_{\bm{x}_{0}}^{\bm{w}}(t) over 𝒳∖𝒳s\mathcal{X}\setminus\mathcal{X}_{s}, and is equal to ∂v​(t,𝒙)∂t\frac{\partial v(t,\bm{x})}{\partial t} on ∂𝒳∪∂𝒳s\partial\mathcal{X}\cup\partial\mathcal{X}_{s} [kushner1967stochastic]. That is, for v​(t,𝒙)v(t,\bm{x}) that is twice continuously differentiable over 𝒙\bm{x} and continuously differentiable over tt,

for (𝒙,t)∈𝒳∖𝒳s×[0,T](\bm{x},t)\in\mathcal{X}\setminus\mathcal{X}_{s}\times[0,T] and

for 𝒙∈∂𝒳∪∂𝒳s\bm{x}\in\partial\mathcal{X}\cup\partial\mathcal{X}_{s} and t∈[0,T]t\in[0,T].

Given an initial state 𝒙0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}, the exact reachability probability ℙ𝒙0[0,T]\mathbb{P}_{\bm{x}_{0}}^{[0,T]} is equal to the probability of reaching the set ∂𝒳s\partial\mathcal{X}_{s} at the time instant TT for the above auxiliary stochastic process. Before justifying this statement, we first show that the reachability probability ℙ𝒙0[0,T]\mathbb{P}_{\bm{x}_{0}}^{[0,T]} is equal to the probability that system (1) will reach the boundary ∂𝒳s\partial\mathcal{X}_{s} of the set 𝒳s\mathcal{X}_{s} within the time horizon [0,T][0,T] while remaining within the state constrained set 𝒳\mathcal{X} until the first occurrence of hitting the set ∂𝒳s\partial\mathcal{X}_{s}.

Lemma 1.

Given 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s},

| ℙ𝒙0[0,T]=ℙ​({𝒘∈Ω|∃t∈[0,T].𝑿𝒙0𝒘​(t)∈∂𝒳s∧∀τ∈[0,t).𝑿𝒙0𝒘​(τ)∈𝒳∖𝒳s}).\mathbb{P}_{\bm{x}_{0}}^{[0,T]}=\mathbb{P}\Bigg(\left\{\bm{w}\in\Omega\,\middle|\,\begin{aligned} &\exists t\in[0,T].\bm{X}_{\bm{x}_{0}}^{\bm{w}}(t)\in\partial\mathcal{X}_{s}\wedge\\ &\forall\tau\in[0,t).\bm{X}_{\bm{x}_{0}}^{\bm{w}}(\tau)\in\mathcal{X}\setminus\mathcal{X}_{s}\end{aligned}\right\}\Bigg). | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |

Proof.

Since 𝒳s\mathcal{X}_{s} is a closed set and the sample path 𝑿𝒙0𝒘​(t)\bm{X}_{\bm{x}_{0}}^{\bm{w}}(t) is continuous over tt, we have A=BA=B, where A={𝒘∈Ω∣∃t∈[0,T].𝑿𝒙0𝒘​(t)∈∂𝒳s​⋀∀τ∈[0,t).𝑿𝒙0𝒘​(τ)∈𝒳}A=\{\,\bm{w}\in\Omega\mid\exists t\in[0,T].\bm{X}_{\bm{x}_{0}}^{\bm{w}}(t)\in\partial\mathcal{X}_{s}\bigwedge\forall\tau\in[0,t).\bm{X}_{\bm{x}_{0}}^{\bm{w}}(\tau)\in\mathcal{X}\,\} and B={𝒘∈Ω∣∃t∈[0,T].𝑿𝒙0𝒘​(t)∈𝒳s​⋀∀τ∈[0,t).𝑿𝒙0𝒘​(τ)∈𝒳}B=\{\,\bm{w}\in\Omega\mid\exists t\in[0,T].\bm{X}_{\bm{x}_{0}}^{\bm{w}}(t)\in\mathcal{X}_{s}\bigwedge\forall\tau\in[0,t).\bm{X}_{\bm{x}_{0}}^{\bm{w}}(\tau)\in\mathcal{X}\,\}. Thus, we have the conclusion. ∎

Lemma 2 (Lemma 1, [xue2023safe]).

Given 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s},

Moreover, for any 0<T1≤T2≤T0<T_{1}\leq T_{2}\leq T,

Further, the exact reachability probability ℙ𝒙0[0,T]\mathbb{P}_{\bm{x}_{0}}^{[0,T]} can be reduced to a solution to a second-order partial differential equation.

Proposition 2.

Suppose there exists a function v​(t,𝐱):[0,T]×ℝnv(t,\bm{x})\colon[0,T]\times\mathbb{R}^{n} which is twice continuously differentiable over 𝐱\bm{x} and continuously differentiable over tt, satisfying

thenℙ𝐱0[0,T]=v​(0,𝐱0)\mathbb{P}_{\bm{x}_{0}}^{[0,T]}=v(0,\bm{x}_{0}) for 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}.

Proof.

The proof relies on Dynkin’s formula and Lemma 2: for 𝒙0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}, we obtain

The proof is completed. ∎

3.1 Upper-bounding Reachability Probabilities

In this subsection, we present our barrier functions for upper-bounding the reachability probability, denoted as ℙ𝒙0[0,T]\mathbb{P}_{\bm{x}_{0}}^{[0,T]} in Definition 2. The first time-dependent barrier function is obtained via relaxing equation (4) as stated in Proposition 2. The second one extends upon the first one, which relaxes the supermartingale requirement. The third one is a variant of the second one, using a time-independent function v​(𝒙)v(\bm{x}) instead of a time-dependent function v​(t,𝒙)v(t,\bm{x}). They are respectively formulated in Corollary 1, Theorem 1, and Corollary 2.

Corollary 1.

Suppose there exists a barrier function v​(t,𝐱):[0,T]×ℝn→ℝv(t,\bm{x})\colon[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R} that is continuously differentiable over tt and twice continuously differentiable over 𝐱\bm{x}, satisfying

then ℙ𝐱0[0,T]≤v​(0,𝐱0)\mathbb{P}_{\bm{x}_{0}}^{[0,T]}\leq v(0,\bm{x}_{0}) for 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}.

Proof.

The first condition in (5), ℒ​v​(t,𝒙)≤0\mathcal{L}v(t,\bm{x})\leq 0 for 𝒙∈𝒳∖𝒳s\bm{x}\in\mathcal{X}\setminus\mathcal{X}_{s}, directly implies ℒ^​v​(t,𝒙)≤0\widehat{\mathcal{L}}v(t,\bm{x})\leq 0 for 𝒙∈𝒳∖𝒳s\bm{x}\in\mathcal{X}\setminus\mathcal{X}_{s}. The second condition in (5), ∂v​(t,𝒙)∂t≤0\frac{\partial v(t,\bm{x})}{\partial t}\leq 0 for 𝒙∈∂𝒳∪∂𝒳s\bm{x}\in\partial\mathcal{X}\cup\partial\mathcal{X}_{s}, directly implies ℒ^​v​(t,𝒙)≤0\widehat{\mathcal{L}}v(t,\bm{x})\leq 0 for 𝒙∈∂𝒳∪∂𝒳s\bm{x}\in\partial\mathcal{X}\cup\partial\mathcal{X}_{s}. Combining these two results with the third condition v​(T,𝒙)≥1∂𝒳s​(𝒙),∀𝒙∈𝒳∖𝒳s¯v(T,\bm{x})\geq 1_{\partial\mathcal{X}_{s}}(\bm{x}),\forall\bm{x}\in\overline{\mathcal{X}\setminus\mathcal{X}_{s}}, we have

Based on (6), we can obtain the conclusion using the Dynkin’s formula and Lemma 2: for 𝒙0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}, we obtain

The proof is completed. ∎

We found another time-dependent barrier function for upper-bounding the reachability probability ℙ𝒙0[0,T]\mathbb{P}_{\bm{x}_{0}}^{[0,T]} in Theorem 5 in [feng2020unbounded], which is formulated below: Suppose there exists a constant η>0\eta>0 and a barrier function v​(t,𝒙):ℝ×ℝnv(t,\bm{x})\colon\mathbb{R}\times\mathbb{R}^{n}, satisfying

then ℙ𝒙0[0,T]≤v​(0,𝒙0)η\mathbb{P}_{\bm{x}_{0}}^{[0,T]}\leq\frac{v(0,\bm{x}_{0})}{\eta}.

Upon comparing conditions (5) and (7), it becomes apparent that condition (7) imposes the requirement of non-negativity for the barrier function v​(t,𝒙)v(t,\bm{x}) over [0,T]×𝒳¯[0,T]\times\overline{\mathcal{X}}. Conversely, condition (5) solely necessitates non-negativity for the function v​(t,𝒙)v(t,\bm{x}) over 𝒳∖𝒳s¯\overline{\mathcal{X}\setminus\mathcal{X}_{s}} at t=Tt=T. These disparities arise due to the construction of (7) using the well-established Doob’s non-negative supermartingale inequality (also known as Ville’s inequality [doob1939jean]). In contrast, condition (5) is formulated by relaxing equation (4).

Corollary 1 states that if there exists a barrier function v​(t,𝒙):[0,T]×ℝn→ℝv(t,\bm{x})\colon[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R} satisfying (5), then the reachability probability ℙ𝒙0[0,T]\mathbb{P}_{\bm{x}_{0}}^{[0,T]} can be bounded above by v​(0,𝒙0)v(0,\bm{x}_{0}). However, the requirement for ℒ​v​(t,𝒙)≤0\mathcal{L}v(t,\bm{x})\leq 0 to hold over (t,𝒙)∈[0,T]×(𝒳∖𝒳s)(t,\bm{x})\in[0,T]\times(\mathcal{X}\setminus\mathcal{X}_{s}) may hinder the acquisition of such a barrier function. In the following, we will further relax this supermartingale requirement.

Theorem 1.

Suppose there exists a barrier function v​(t,𝐱):[0,T]×ℝn→ℝv(t,\bm{x})\colon[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R} that is continuously differentiable over tt and twice continuously differentiable over 𝐱\bm{x}, satisfying

then

for 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}.

Proof.

From (8), we have

If α=0\alpha=0, we have that

When α≠0\alpha\neq 0, we have, by the Grönwall’s inequality in the differential form, that

Also, since v​(T,𝒙)≥1∂𝒳s​(𝒙),∀𝒙∈𝒳∖𝒳s¯v(T,\bm{x})\geq 1_{\partial\mathcal{X}_{s}}(\bm{x}),\forall\bm{x}\in\overline{\mathcal{X}\setminus\mathcal{X}_{s}}, we have ℙ𝒙0[0,T]=𝔼​[1∂𝒳s​(𝑿^𝒙0𝒘​(T))]≤𝔼​[v​(T,𝑿^𝒙0𝒘​(T))]\mathbb{P}_{\bm{x}_{0}}^{[0,T]}=\mathbb{E}[1_{\partial\mathcal{X}_{s}}(\widehat{\bm{X}}_{\bm{x}_{0}}^{\bm{w}}(T))]\leq\mathbb{E}[v(T,\widehat{\bm{X}}_{\bm{x}_{0}}^{\bm{w}}(T))] and thus ℙ𝒙0[0,T]=ℙ​(𝑿^𝒙0𝒘​(T)∈∂𝒳s)≤eα​T​v​(0,𝒙0)+βα​(eα​T−1)\mathbb{P}_{\bm{x}_{0}}^{[0,T]}=\mathbb{P}(\widehat{\bm{X}}_{\bm{x}_{0}}^{\bm{w}}(T)\in\partial\mathcal{X}_{s})\leq e^{\alpha T}v(0,\bm{x}_{0})+\frac{\beta}{\alpha}(e^{\alpha T}-1).

The proof is completed. ∎

It is easy to observe that condition (5) in Corollary 1 is a special case of the one (8) in Theorem 1 with α=β=0\alpha=\beta=0. Below, a straightforward variant of the result in Theorem 1 is presented. Instead of resorting to a time-dependent barrier function v​(t,𝒙):[0,T]×ℝn→ℝv(t,\bm{x})\colon[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R}, the focus is on finding a time-independent one v​(𝒙):ℝn→ℝv(\bm{x})\colon\mathbb{R}^{n}\rightarrow\mathbb{R}.

Corollary 2.

Suppose there exists a barrier function v​(𝐱):ℝn→ℝv(\bm{x})\colon\mathbb{R}^{n}\rightarrow\mathbb{R}, which is twice continuously differentiable, satisfying

then

for 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}.

From v​(𝒙)≥1∂𝒳s​(𝒙),∀𝒙∈𝒳∖𝒳s¯v(\bm{x})\geq 1_{\partial\mathcal{X}_{s}}(\bm{x}),\forall\bm{x}\in\overline{\mathcal{X}\setminus\mathcal{X}_{s}} in (9), we can obtain v​(𝒙)≥0v(\bm{x})\geq 0 for 𝒙∈∂𝒳\bm{x}\in\partial\mathcal{X} and v​(𝒙)≥1v(\bm{x})\geq 1 for 𝒙∈∂𝒳s\bm{x}\in\partial\mathcal{X}_{s}. Since 0≤α​v​(𝒙)+β,∀𝒙∈∂𝒳∪∂𝒳s0\leq\alpha v(\bm{x})+\beta,\forall\bm{x}\in\partial\mathcal{X}\cup\partial\mathcal{X}_{s}, we can obtain α+β≥0\alpha+\beta\geq 0 if α≤0\alpha\leq 0. However, it is worth remarking here that α>0\alpha>0 is permitted in condition (9).

Another upper bound of ℙ𝒙0[0,T]\mathbb{P}_{\bm{x}_{0}}^{[0,T]} was derived in [santoyo2021barrier]: if v​(𝒙)v(\bm{x}) satisfies

an upper bound of ℙ𝒙0[0,T]\mathbb{P}_{\bm{x}_{0}}^{[0,T]} is

These results are obtained via following Theorem 1 in Chapter 3 in [kushner1967stochastic] and are built upon the known Doob’s nonnegative supermartingale inequality (or, Ville’s inequality [doob1939jean]) as condition (7). However, it is observed that as α\alpha approaches 0−0^{-}, (v​(𝒙0)−(eβ​T−1)​βα)​e−β​T(v(\bm{x}_{0})-(e^{\beta T}-1)\frac{\beta}{\alpha})e^{-\beta T} is not equal to v​(𝒙0)+β​Tv(\bm{x}_{0})+\beta T as expected. In contrast, it tends to infinity, which is overly conservative. Condition (9) in Corollary 2 has certain advantages over (10): firstly, as α\alpha approaches 0−0^{-}, the expression eα​T​v​(𝒙0)+βα​(eα​T−1)e^{\alpha T}v(\bm{x}_{0})+\frac{\beta}{\alpha}(e^{\alpha T}-1) converges to v​(𝒙0)+β​Tv(\bm{x}_{0})+\beta T; secondly, when v​(𝒙0)≤1<−βαv(\bm{x}_{0})\leq 1<-\frac{\beta}{\alpha} (when α<0\alpha<0, 1<−βα1<-\frac{\beta}{\alpha} implies α+β>0\alpha+\beta>0), we can obtain (v​(𝒙0)−(eβ​T−1)​βα)​e−β​T>eα​T​v​(𝒙0)+βα​(eα​T−1)(v(\bm{x}_{0})-(e^{\beta T}-1)\frac{\beta}{\alpha})e^{-\beta T}>e^{\alpha T}v(\bm{x}_{0})+\frac{\beta}{\alpha}(e^{\alpha T}-1). On the other hand, it can be observed that, when v​(𝒙0)>1v(\bm{x}_{0})>1, α<0\alpha<0, α+β>0\alpha+\beta>0, and T>0T>0, the upper bound (v​(𝒙0)−(eβ​T−1)​βα)​e−β​T\bigl(v(\bm{x}_{0})-(e^{\beta T}-1)\tfrac{\beta}{\alpha}\bigr)e^{-\beta T} is always greater than 1 and is therefore meaningless. Consequently, under the condition α<0\alpha<0 and α+β>0\alpha+\beta>0, a constraint v​(𝒙0)≤1v(\bm{x}_{0})\leq 1 should be added in (10) in order to make the upper bound (v​(𝒙0)−(eβ​T−1)​βα)​e−β​T\bigl(v(\bm{x}_{0})-(e^{\beta T}-1)\frac{\beta}{\alpha}\bigr)e^{-\beta T} be meaningful (i.e., be less than or equal to 1); fourthly, in the case α=0\alpha=0, condition (9) is strictly weaker than (10), because it relaxes the requirements that v​(𝒙)v(\bm{x}) remain nonnegative throughout 𝒳\mathcal{X} and that v​(𝒙)≥1v(\bm{x})\geq 1 hold over the entire 𝒳s\mathcal{X}_{s}. Finally, unlike the aforementioned condition (10), condition (9) is not restricted to the scenario where α≤0\alpha\leq 0 and/or β≥0\beta\geq 0.

3.2 Lower-bounding Reachability Probabilities

In this subsection, we present our barrier functions for lower-bounding the reachability probability ℙ𝒙0[0,T]\mathbb{P}_{\bm{x}_{0}}^{[0,T]}.

The construction of the first barrier function was inspired by [xue2021reach, Xue2023]. It cannot be obtained via relaxing equation (4) directly. An auxiliary function is introduced.

Proposition 3.

Suppose there exist a barrier function v​(t,𝐱):[0,T]×ℝn→ℝv(t,\bm{x})\colon[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R} and a function w​(t,𝐱):[0,T]×ℝn→ℝw(t,\bm{x})\colon[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R} with sup(t,𝐱)∈[0,T]×𝒳¯|w​(t,𝐱)|≤M\sup_{(t,\bm{x})\in[0,T]\times\overline{\mathcal{X}}}|w(t,\bm{x})|\leq M that are continuously differentiable over tt and twice continuously differentiable over 𝐱\bm{x}, satisfying

then,ℙ𝐱0[0,T]≥v​(0,𝐱0)−2​MT\mathbb{P}_{\bm{x}_{0}}^{[0,T]}\geq v(0,\bm{x}_{0})-\frac{2M}{T} for 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}.

Proof.

From (11), we have

According to ℒ^​v​(t,𝒙)≥0,∀𝒙∈𝒳∖𝒳s¯,∀t∈[0,T]\widehat{\mathcal{L}}v(t,\bm{x})\geq 0,\forall\bm{x}\in\overline{\mathcal{X}\setminus\mathcal{X}_{s}},\forall t\in[0,T], we have, for t∈[0,T]t\in[0,T], that

Further, from v​(t,𝒙)≤1∂𝒳s​(𝒙)+ℒ^​w​(t,𝒙),∀𝒙∈𝒳∖𝒳s¯,∀t∈[0,T]v(t,\bm{x})\leq 1_{\partial\mathcal{X}_{s}}(\bm{x})+\widehat{\mathcal{L}}w(t,\bm{x}),\forall\bm{x}\in\overline{\mathcal{X}\setminus\mathcal{X}_{s}},\forall t\in[0,T], we have

The proof is completed. ∎

Similar to Theorem 1, a barrier function that relaxes the submartingale requirement (i.e., ℒ​v​(t,𝒙)≥0,∀𝒙∈𝒳∖𝒳s,∀t∈[0,T]\mathcal{L}v(t,\bm{x})\geq 0,\forall\bm{x}\in\mathcal{X}\setminus\mathcal{X}_{s},\forall t\in[0,T]) in Proposition 3 is formulated in Theorem 2.

Theorem 2.

Suppose there exist a barrier function v​(t,𝐱):[0,T]×ℝn→ℝv(t,\bm{x})\colon[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R} and a function w​(t,𝐱):[0,T]×ℝn→ℝw(t,\bm{x})\colon[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R} with sup(t,𝐱)∈[0,T]×𝒳¯|w​(t,𝐱)|≤M\sup_{(t,\bm{x})\in[0,T]\times\overline{\mathcal{X}}}|w(t,\bm{x})|\leq M that are continuously differentiable over tt and twice continuously differentiable over 𝐱\bm{x}, satisfying

then

for 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}.

Proof.

From (12), we have

When α≠0\alpha\neq 0, according to ℒ^​v​(t,𝒙)≥0,∀𝒙∈𝒳∖𝒳s¯,∀t∈[0,T]\widehat{\mathcal{L}}v(t,\bm{x})\geq 0,\forall\bm{x}\in\overline{\mathcal{X}\setminus\mathcal{X}_{s}},\forall t\in[0,T], we have, for t∈[0,T]t\in[0,T], that

Further, from v​(t,𝒙)≤1∂𝒳s​(𝒙)+ℒ^​w​(t,𝒙),∀t∈[0,T],∀𝒙∈𝒳∖𝒳s¯v(t,\bm{x})\leq 1_{\partial\mathcal{X}_{s}}(\bm{x})+\widehat{\mathcal{L}}w(t,\bm{x}),\forall t\in[0,T],\forall\bm{x}\in\overline{\mathcal{X}\setminus\mathcal{X}_{s}}, we have

The conclusion for α=0\alpha=0 can be obtained via following the above procedure. The proof is completed. ∎

Further, a straightforward result can be obtained from Theorem 2 when searching for a time-independent function v​(𝒙):ℝn→ℝv(\bm{x})\colon\mathbb{R}^{n}\rightarrow\mathbb{R} instead of a time-dependent function v​(t,𝒙):[0,T]×ℝn→ℝv(t,\bm{x})\colon[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R}.

Corollary 3.

Suppose there exist twice continuously differentiable functions v​(𝐱):ℝn→ℝv(\bm{x})\colon\mathbb{R}^{n}\rightarrow\mathbb{R} and w​(𝐱):ℝn→ℝw(\bm{x})\colon\mathbb{R}^{n}\rightarrow\mathbb{R} with sup𝐱∈𝒳¯|w​(𝐱)|≤M\sup_{\bm{x}\in\overline{\mathcal{X}}}|w(\bm{x})|\leq M, satisfying

then

for 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}.

Besides, we can also obtain similar conclusions as in Remark 4 and 5.

4 Bounding Reachability Probabilities II

This section introduces our barrier functions for upper- and lower-bounding the reachability probability ℙ𝒙0T\mathbb{P}_{\bm{x}_{0}}^{T} in Definition 3. They are respectively formulated in Subsection 4.1 and 4.2.

Since the event for reachability at a specific time instant (Definition 3) is fundamentally different from the one for a time horizon (Definition 2)—specifically, A={𝒘∈Ω∣𝑿𝒙0𝒘​(T)∈𝒳s∧∀τ∈[0,T).𝑿𝒙0𝒘​(τ)∈𝒳}≠{𝒘∈Ω∣𝑿𝒙0𝒘​(T)∈∂𝒳s∧∀τ∈[0,T).𝑿𝒙0𝒘​(τ)∈𝒳}=BA=\{\,\bm{w}\in\Omega\mid\bm{X}_{\bm{x}_{0}}^{\bm{w}}(T)\in\mathcal{X}_{s}\wedge\forall\tau\in[0,T).\bm{X}_{\bm{x}_{0}}^{\bm{w}}(\tau)\in\mathcal{X}\,\}\neq\{\,\bm{w}\in\Omega\mid\bm{X}_{\bm{x}_{0}}^{\bm{w}}(T)\in\partial\mathcal{X}_{s}\wedge\forall\tau\in[0,T).\bm{X}_{\bm{x}_{0}}^{\bm{w}}(\tau)\in\mathcal{X}\,\}=B—the construction of the barrier functions in this section lies on a different auxiliary stochastic process {𝑿~𝒙0𝒘​(t),t∈ℝ≥0}\{\widetilde{\bm{X}}_{\bm{x}_{0}}^{\bm{w}}(t),t\in\mathbb{R}_{\geq 0}\} for 𝒙0∈𝒳∖𝒳s¯\bm{x}_{0}\in\overline{\mathcal{X}\setminus\mathcal{X}_{s}} that is a stopped process corresponding to {𝑿𝒙0𝒘​(t),t∈[0,T𝒙0​(𝒘))}\{\bm{X}_{\bm{x}_{0}}^{\bm{w}}(t),t\in[0,T^{\bm{x}_{0}}(\bm{w}))\} and the set 𝒳¯\overline{\mathcal{X}} rather than 𝒳∖𝒳s¯\overline{\mathcal{X}\setminus\mathcal{X}_{s}} as {𝑿^𝒙0𝒘​(t),t∈[0,T𝒙0​(𝒘))}\{\widehat{\bm{X}}_{\bm{x}_{0}}^{\bm{w}}(t),t\in[0,T^{\bm{x}_{0}}(\bm{w}))\}, i.e.,

where τ𝒙0​(𝒘)=inf{t∣𝑿𝒙0𝒘​(t)∈∂𝒳}\tau^{\bm{x}_{0}}(\bm{w})=\inf\{\,t\mid\bm{X}_{\bm{x}_{0}}^{\bm{w}}(t)\in\partial\mathcal{X}\,\} is the first time of exit of 𝑿𝒙0𝒘​(t)\bm{X}_{\bm{x}_{0}}^{\bm{w}}(t) from the set 𝒳\mathcal{X}. Similar to the stopped process in Section 3, the infinitesimal generator ℒ~\widetilde{\mathcal{L}} on a test function v​(t,𝒙)v(t,\bm{x}) corresponding to 𝑿~𝒙0𝒘​(t)\widetilde{\bm{X}}_{\bm{x}_{0}}^{\bm{w}}(t) is identical to the one corresponding to 𝑿𝒙0𝒘​(t)\bm{X}_{\bm{x}_{0}}^{\bm{w}}(t) over 𝒳\mathcal{X}, and is equal to ∂v​(t,𝒙)∂t\frac{\partial v(t,\bm{x})}{\partial t} on ∂𝒳\partial\mathcal{X} [kushner1967stochastic]. That is, for v​(t,𝒙)v(t,\bm{x}) which is twice continuously differentiable over 𝒙\bm{x} and continuously differentiable over tt,

for (𝒙,t)∈𝒳×[0,T](\bm{x},t)\in\mathcal{X}\times[0,T] and ℒ~​v​(t,𝒙)=∂v​(t,𝒙)∂t\widetilde{\mathcal{L}}v(t,\bm{x})=\frac{\partial v(t,\bm{x})}{\partial t} for 𝒙∈∂𝒳\bm{x}\in\partial\mathcal{X} and t∈[0,T]t\in[0,T].

The exact reachability probability ℙ𝒙0T\mathbb{P}_{\bm{x}_{0}}^{T} is equal to the probability of reaching the set 𝒳s\mathcal{X}_{s} at the time instant TT for the above auxiliary stochastic process.

Lemma 3.

Given 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s},

The reachability probability ℙ𝒙0T\mathbb{P}_{\bm{x}_{0}}^{T} can also be reduced to a solution to a second-order partial differential equation.

Proposition 4.

Suppose there exists a function v​(t,𝐱):[0,T]×ℝnv(t,\bm{x})\colon[0,T]\times\mathbb{R}^{n} which is twice continuously differentiable over 𝐱\bm{x} and continuously differentiable over tt, satisfying

thenℙ𝐱0T=ℙ​(𝐗~𝐱0𝐰​(T)∈𝒳s)=v​(0,𝐱0)\mathbb{P}_{\bm{x}_{0}}^{T}=\mathbb{P}(\widetilde{\bm{X}}_{\bm{x}_{0}}^{\bm{w}}(T)\in\mathcal{X}_{s})=v(0,\bm{x}_{0}) for 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}.

Proof.

The proof is similar to Proposition 2. ∎

4.1 Upper-bounding Reachability Probabilities

In this subsection, we present barrier functions for upper-bounding the reachability probability ℙ𝒙0T\mathbb{P}_{\bm{x}_{0}}^{T} in Definition 3. The first time-dependent barrier function is obtained via relaxing equation (17) in Proposition 4. The second one extends upon the first one, which relaxes the supermartingale requirement. The third one is a variant of the second one, using a time-independent function v​(𝒙)v(\bm{x}) instead of a time-dependent one v​(t,𝒙)v(t,\bm{x}). They are respectively formulated in Corollary 4, Theorem 3, and Corollary 5.

Corollary 4.

Suppose there exists a barrier function v​(t,𝐱):[0,T]×ℝn→ℝv(t,\bm{x})\colon[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R} that is continuously differentiable over tt and twice continuously differentiable over 𝐱\bm{x}, satisfying

thenℙ𝐱0T≤v​(0,𝐱0)\mathbb{P}_{\bm{x}_{0}}^{T}\leq v(0,\bm{x}_{0}) for 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}.

Proof.

From (18), we have

Then, we can obtain the conclusion via using Lemma 3 and following the proof of Corollary 1. ∎

Corollary 4 states that if there exists a barrier function v​(t,𝒙):[0,T]×ℝn→ℝv(t,\bm{x})\colon[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R} satisfying (18), then the reachability probability ℙ𝒙0T\mathbb{P}_{\bm{x}_{0}}^{T} can be bounded above by v​(0,𝒙0)v(0,\bm{x}_{0}). Analogously, the requirement for ℒ​v​(t,𝒙)≤0\mathcal{L}v(t,\bm{x})\leq 0 to hold for (t,𝒙)∈[0,T]×𝒳(t,\bm{x})\in[0,T]\times\mathcal{X} may hinder the acquisition of such a barrier function. We will relax this requirement below.

Theorem 3.

Suppose there exists a barrier function v​(t,𝐱):[0,T]×ℝn→ℝv(t,\bm{x})\colon[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R} that is continuously differentiable over tt and twice continuously differentiable over 𝐱\bm{x}, satisfying

then

for 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}.

Proof.

From (20), we have

Then, we can obtain the conclusion via using Lemma 3 and following the proof of Corollary 1. ∎

A straightforward result is obtained from Theorem 3 when searching for a time-independent barrier function v​(𝒙):ℝn→ℝv(\bm{x})\colon\mathbb{R}^{n}\rightarrow\mathbb{R} instead of a time-dependent one v​(t,𝒙):[0,T]×ℝn→ℝv(t,\bm{x})\colon[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R}.

Corollary 5.

Suppose there exists a barrier function v​(𝐱):ℝn→ℝv(\bm{x})\colon\mathbb{R}^{n}\rightarrow\mathbb{R}, which is twice continuously differentiable, satisfying

then

for 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}.

4.2 Lower-bounding Reachability Probabilities

In this subsection, we present our barrier functions for lower-bounding the reachability probability ℙ𝒙0T\mathbb{P}_{\bm{x}_{0}}^{T}, based on the following assumption on 𝒳s\mathcal{X}_{s}.

Assumption 1.

The set 𝒳s\mathcal{X}_{s} has non-empty interior, i.e., 𝒳s∘≠∅\mathcal{X}_{s}^{\circ}\neq\emptyset.

This assumption is necessary to ensure the possibility of a non-trivial (positive) lower bound on ℙ𝒙0T\mathbb{P}_{\bm{x}_{0}}^{T} in the following constructed barrier functions. The conditions for lower-bounding ℙ𝒙0T\mathbb{P}_{\bm{x}_{0}}^{T} require a barrier function v​(t,𝒙)v(t,\bm{x}) to satisfy v​(T,𝒙)≤𝟏𝒳s​(𝒙)v(T,\bm{x})\leq\mathbf{1}_{\mathcal{X}_{s}}(\bm{x}) for all 𝒙∈𝒳¯\bm{x}\in\overline{\mathcal{X}}.

• 1. If 𝒳s\mathcal{X}_{s} has no interior (e.g., it is a single point or a lower-dimensional manifold), then 1𝒳s​(𝒙)=01_{\mathcal{X}_{s}}(\bm{x})=0 for almost every 𝒙∈𝒳¯\bm{x}\in\overline{\mathcal{X}}.
• 1. Given the continuity of v​(T,𝒙)v(T,\bm{x}), the condition would then simplify to v​(T,𝒙)≤0v(T,\bm{x})\leq 0 everywhere.
• 1. This, in combination with the generator conditions, would typically force the obtained lower bounds to be non-positive.

Like the one in Corollary 4, the construction of the first barrier function was obtained by relaxing equation (17).

Corollary 6.

Suppose there exists a barrier function v​(t,𝐱):[0,T]×ℝn→ℝv(t,\bm{x})\colon[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R} that is continuously differentiable over tt and twice continuously differentiable over 𝐱\bm{x}, satisfying

thenℙ𝐱0T≥v​(0,𝐱0)\mathbb{P}_{\bm{x}_{0}}^{T}\geq v(0,\bm{x}_{0}) for 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}.

Proof.

From (22), we have

Then, we can obtain the conclusion via using Lemma 3 and following the proof of Corollary 1. ∎

Based on the stochastic process 𝑿~𝒙0𝒘​(⋅)\widetilde{\bm{X}}_{\bm{x}_{0}}^{\bm{w}}(\cdot), one might wander the possibility of constructing a barrier function, similar to the one in Corollary 6, to lower-bound ℙ𝒙0T\mathbb{P}_{\bm{x}_{0}}^{T}. Let us consider the existence of a barrier function v​(t,𝒙):[0,T]×ℝn→ℝv(t,\bm{x})\colon[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R} that is twice continuously differentiable with respect to 𝒙\bm{x} and continuously differentiable over tt, satisfying

We can conclude that ℙ𝒙0[0,T]≥v​(0,𝒙0)\mathbb{P}_{\bm{x}_{0}}^{[0,T]}\geq v(0,\bm{x}_{0}) for 𝒙0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}. However, it is observed that if v​(𝒙)v(\bm{x}) satisfies (23), it also satisfies (11) with w​(𝒙)≡0w(\bm{x})\equiv 0. As commented in Remark 1, we cannot obtain meaningful results.

Similar to Theorem 3, a condition that relaxes the submartingale requirement (i.e., ℒ​v​(t,𝒙)≥0,∀𝒙∈𝒳,∀t∈[0,T]\mathcal{L}v(t,\bm{x})\geq 0,\forall\bm{x}\in\mathcal{X},\forall t\in[0,T]) in Proposition 6 is formulated in Theorem 4.

Theorem 4.

Suppose there exist a barrier function v​(t,𝐱):[0,T]×ℝn→ℝv(t,\bm{x})\colon[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R} that is continuously differentiable over tt and twice continuously differentiable over 𝐱\bm{x}, satisfying

then

for 𝐱0∈𝒳∖𝒳s\bm{x}_{0}\in\mathcal{X}\setminus\mathcal{X}_{s}.

Proof.

From (24), we have

If α≠0\alpha\neq 0, according to ℒ~​v​(t,𝒙)≥α​v​(t,𝒙)+β,∀𝒙∈𝒳¯,∀t∈[0,T]\widetilde{\mathcal{L}}v(t,\bm{x})\geq\alpha v(t,\bm{x})+\beta,\forall\bm{x}\in\overline{\mathcal{X}},\forall t\in[0,T], we have, for t∈[0,T]t\in[0,T], that

Further, from v​(T,𝒙)≤1∂𝒳s​(𝒙),∀𝒙∈𝒳¯v(T,\bm{x})\leq 1_{\partial\mathcal{X}_{s}}(\bm{x}),\forall\bm{x}\in\overline{\mathcal{X}}, we have

If α=0\alpha=0, we can obtain ℙ𝒙0T≥v​(0,𝒙0)+β​T\mathbb{P}_{\bm{x}_{0}}^{T}\geq v(0,\bm{x}_{0})+\beta T by following the above procedure.

The proof is completed. ∎

5 Examples

In this section, we assess the effectiveness of the proposed barrier certificates in bounding reachability probabilities through numerical examples. As a benchmark, we compute empirical estimates of the exact reachability probabilities using 10410^{4} Monte Carlo simulations with the Euler–Maruyama method. All conditions (7), (8), (9),(10), (12), (13), (20), (21), and (24) are encoded into semidefinite programs via SOS decomposition [papachristodoulou2005tutorial], solved with MOSEK 10.2 [aps2019mosek]. To avoid the bilinearity that would arise from jointly optimizing over both the parameter α\alpha and the barrier function vv, we adopt a pragmatic strategy: we carry out a simple one-dimensional exploration by manually trying different values of α\alpha. For each fixed α\alpha, the problem of finding a suitable barrier function v​(t,𝒙)v(t,\bm{x}) or v​(𝒙)v(\bm{x}) becomes a convex optimization problem. Unless otherwise stated, we utilize polynomial barrier functions of degree dd, meaning they include all monomials with total degree less than or equal to dd.

Example 1 (Population Growth Model).

Consider the following system:

with state constraint 𝒳={x∈ℝ∣x2−1<0}\mathcal{X}=\{x\in\mathbb{R}\mid x^{2}-1<0\} and safe set 𝒳s={x∈ℝ∣100​x2−1≤0}\mathcal{X}_{s}=\{x\in\mathbb{R}\mid 100x^{2}-1\leq 0\}.

Case 1. b​(x)=−xb(x)=-x, σ​(x)=22​x\sigma(x)=\tfrac{\sqrt{2}}{2}x, T=100T=100, x0=−0.8x_{0}=-0.8. Fig. 1 illustrates trajectories in the xx–tt space. Computed bounds are summarized in Table 1.

Figure 1: Case 1 of Example 1. The region bounded by the red curve represents the safe set over the horizon [0,100][0,100], the region bounded by the green curve represents the target set over the horizon [0,100][0,100], and the blue curves represent five simulated trajectories starting from the initial state x0=−0.8x_{0}=-0.8.

Table 1: Bounds for Example 1, Case 1 (b​(x)=−xb(x)=-x, σ​(x)=22​x\sigma(x)=\tfrac{\sqrt{2}}{2}x, T=100T=100, x0=−0.8x_{0}=-0.8).

Remark. The bounds obtained from time-independent conditions are already tight; time-dependent barrier functions give comparable results but sometimes suffer from numerical instability.

Case 2. b​(x)=−10​xb(x)=-10x, σ​(x)=22​x\sigma(x)=\tfrac{\sqrt{2}}{2}x, T=1T=1, x0=−0.8x_{0}=-0.8. Fig. 2 illustrates trajectories in the xx–tt space. Table 2 shows the computed bounds.

Figure 2: Case 2 of Example 1. The region bounded by the red curve represents the safe set over the horizon [0,1][0,1], the region bounded by the green curve represents the target set over the horizon [0,1][0,1], and the blue curves represent five simulated trajectories starting from the initial state x0=−0.8x_{0}=-0.8.

Table 2: Bounds for Example 1, Case 2 (b​(x)=−10​xb(x)=-10x, σ​(x)=22​x\sigma(x)=\tfrac{\sqrt{2}}{2}x, T=1T=1, x0=−0.8x_{0}=-0.8).

_Remark._While the time-dependent conditions (12) and (13) provided valid lower bounds (0.72530.7253 and 0.49120.4912 with α=0\alpha=0), even tighter lower bounds for ℙx0[0,T]\mathbb{P}_{x_{0}}^{[0,T]} were obtained by using the barrier function from (24) designed for ℙx0T\mathbb{P}_{x_{0}}^{T}, yielding a bound of 0.90520.9052 (α=0\alpha=0) and 0.93490.9349 (α=1.0\alpha=1.0). This conclusion applies to the case with α=1\alpha=1 as well. This illustrates the potential for cross-application of the barrier certificates between the two reachability problems to achieve sharper results.

Case 3. b​(x)=−x+0.1b(x)=-x+0.1, σ​(x)=x2\sigma(x)=x^{2}, T=10T=10, x0=−0.5x_{0}=-0.5. Fig. 3 illustrates trajectories in the xx–tt space. Table 3 shows the computed bounds.

Figure 3: Case 3 of Example 1. The region bounded by the red curve represents the safe set over the horizon [0,10][0,10], the region bounded by the green curve represents the target set over the horizon [0,10][0,10], and the blue curves represent five simulated trajectories starting from the initial state x0=−0.5x_{0}=-0.5.

Table 3: Bounds for Example 1, Case 3 (b​(x)=−x+0.1b(x)=-x+0.1, σ​(x)=x2\sigma(x)=x^{2}, T=10T=10, x0=−0.5x_{0}=-0.5).

Remark. For this nonlinear system, increasing the degree and adjusting α\alpha improves tightness, though numerical issues appear for higher-degree time-dependent barriers.

Example 2.

Consider the following two-dimensional system:

with 𝒳={x2+y2≤1}\mathcal{X}=\{x^{2}+y^{2}\leq 1\}, 𝒳s={x2+y2≤0.01}\mathcal{X}_{s}=\{x^{2}+y^{2}\leq 0.01\}, x0=(−0.5,0.5)x_{0}=(-0.5,0.5), T=1T=1.

Fig. 4 illustrates trajectories in the xx–yy space. Computed bounds are shown in Table 4.

Figure 4: Example 2. The region bounded by the red curve represents the safe set, the region bounded by the green curve represents the target set, and the blue curves represent five simulated trajectories starting from the initial state (x0,y0)⊤=(−0.5,0.5)⊤(x_{0},y_{0})^{\top}=(-0.5,0.5)^{\top}.

Table 4: Bounds for Example 2.

_Remark._Using time-dependent barrier functions (condition (8), particularly with α=3.0\alpha=3.0) yields tighter upper bounds than the time-independent counterpart (condition (9)) or existing methods (conditions (7) and (10) with α=0\alpha=0). For condition (10) with α<0\alpha<0, we did not obtain any meaningful upper bounds. In addition, the barrier in (20) with α=2.0\alpha=2.0, designed for the specific time-instant problem, provide the tightest upper bound 0.06700.0670 for ℙ𝐱0T\mathbb{P}_{\bm{x}_{0}}^{T}, which is close to the empirical estimate 0 obtained from the Monte Carlo method.

The above examples demonstrate that:

• 1. Comparative Advantage: The new conditions often provide tighter bounds (e.g., Example 2) and more flexible stability ranges for parameters (e.g., α≠0\alpha\neq 0) compared to previous methods, e.g., (7) and (10).
• 1. Cross-Application: Barrier functions designed for one type of reachability probability (e.g., ℙ𝒙0T\mathbb{P}_{\bm{x}_{0}}^{T}) can sometimes be effectively used to bound the other type (e.g., ℙ𝒙0[0,T]\mathbb{P}_{\bm{x}_{0}}^{[0,T]}), providing a useful strategy for obtaining tighter results.
• 1. Parameter Tuning: The parameter α\alpha in the conditions provides a degree of freedom that can be tuned to optimize the tightness of the bounds for a given system.
• 1. Computational Trade-offs: Time-dependent barrier functions can yield sharper results but introduce an additional variable tt, which increases the SDP size. This increased dimensionality often causes numerical instability for higher polynomial degrees, suggesting a practical trade-off between bound tightness and computational tractability.

6 Conclusion

This paper introduced a new framework of constructing barrier functions to establish both upper and lower bounds on the probabilities of reaching specific sets over finite time horizons and at finite time instants in continuous-time stochastic systems described by SDEs. These proposed barrier functions offer stronger alternatives, complement existing methods, or fill gaps, facilitating the calculation of precise bounds on reachability probabilities.

In future work, we will develop advanced numerical methods for computing the proposed barrier functions. Moreover, this framework provides a unifying foundation for reachability analysis, as the stochastic barrier conditions naturally reduce to their deterministic counterparts when the diffusion term vanishes (see Appendix 6.2). A more detailed comparison with specialized deterministic methods will also be pursued.

Acknowledgements

This work is funded by the CAS Pioneer Hundred Talents Program, Basic Research Program of Institute of Software, CAS (Grant No. ISCAS-JCMS-202302), and NRF RSS Scheme NRF-RSS2022-009.

Appendix

6.1 Semi-definite Programming Implementation

In this section we present the semi-definite programs to optimize the lower and upper bounds of the reachability probabilities ℙ𝒙0[0,T]\mathbb{P}_{\bm{x}_{0}}^{[0,T]} and ℙ𝒙0T\mathbb{P}_{\bm{x}_{0}}^{T}. These semi-definite programs are constructed via encoding constraints (7), (8), (9), (12), (13), (20), (5), and (25) into semi-definite constraints using sum of squares decomposition techniques for multivariate polynomials [papachristodoulou2005tutorial].

The following notations are used: ℝ​[𝒙]\mathbb{R}[\bm{x}] denotes the ring of all multivariate polynomials in a variable 𝒙\bm{x}; ∑[𝒙]\sum[\bm{x}] is used to represent the set of sum of squares polynomials over variables𝒙\bm{x}, i.e., ∑[𝒙]={p​(𝒙)∈ℝ​[𝒙]∣p​(𝒙)=∑i=1kqi2​(𝒙),qi∈ℝ​[𝒙],i=1,…,k}\sum[\bm{x}]=\{p(\bm{x})\in\mathbb{R}[\bm{x}]\mid p(\bm{x})=\sum_{i=1}^{k}q_{i}^{2}(\bm{x}),q_{i}\in\mathbb{R}[\bm{x}],i=1,\ldots,k\}. In addition, we assume 𝒳={𝒙∈ℝn∣h​(𝒙)≤0}\mathcal{X}=\{\bm{x}\in\mathbb{R}^{n}\mid h(\bm{x})\leq 0\} with h​(𝒙)∈ℝ​[𝒙]h(\bm{x})\in\mathbb{R}[\bm{x}], and𝒳s={𝒙∈ℝn∣g​(𝒙)≤0}\mathcal{X}_{s}=\{\bm{x}\in\mathbb{R}^{n}\mid g(\bm{x})\leq 0\} with g​(𝒙)∈ℝ​[𝒙]g(\bm{x})\in\mathbb{R}[\bm{x}].

The semidefinite program for solving (7) is:

The above semi-definite program is minimizing the objective via searching (v​(t,𝒙),p​(t,𝒙)∈ℝ​[t,𝒙];si​(t,𝒙)∈∑[t,𝒙],i=0,…,8)(v(t,\bm{x}),p(t,\bm{x})\in\mathbb{R}[t,\bm{x}];s_{i}(t,\bm{x})\in\sum[t,\bm{x}],i=0,\ldots,8).

The semidefinite program for solving (8) is formulated below:

where o​b​j=v​(0,𝒙0)+β​T​if​α=0obj=v(0,\bm{x}_{0})+\beta T~\text{if}~\alpha=0 and o​b​j=eα​T​v​(0,𝒙0)+βα​(eα​T−1)​if​α≠0obj=e^{\alpha T}v(0,\bm{x}_{0})+\frac{\beta}{\alpha}(e^{\alpha T}-1)~\text{if}~\alpha\neq 0, and α\alpha is a user-specified value. The above semi-definite program is minimizing the objective via searching (β;v​(t,𝒙)∈ℝ​[t,𝒙];si​(𝒙)∈∑[t,𝒙],i=0,…,4;p​(𝒙),q​(𝒙),r​(𝒙)∈ℝ​[𝒙];sj​(𝒙)∈∑[𝒙],j=5,…,6)(\beta;v(t,\bm{x})\in\mathbb{R}[t,\bm{x}];s_{i}(\bm{x})\in\sum[t,\bm{x}],i=0,\ldots,4;p(\bm{x}),q(\bm{x}),r(\bm{x})\in\mathbb{R}[\bm{x}];s_{j}(\bm{x})\in\sum[\bm{x}],j=5,\ldots,6).

The semidefinite program for solving (9) is formulated below:

where o​b​j=v​(𝒙0)+β​T​if​α=0obj=v(\bm{x}_{0})+\beta T~\text{if}~\alpha=0 and o​b​j=eα​T​v​(𝒙0)+βα​(eα​T−1)​if​α≠0obj=e^{\alpha T}v(\bm{x}_{0})+\frac{\beta}{\alpha}(e^{\alpha T}-1)~\text{if}~\alpha\neq 0, and α\alpha is a user-specified value. This semi-definite program is minimizing the objective via searching (β;si​(𝒙)∈∑[𝒙],i=0,…,4;v​(𝒙),p​(𝒙),q​(𝒙),r​(𝒙)∈ℝ​[𝒙])(\beta;s_{i}(\bm{x})\in\sum[\bm{x}],i=0,\ldots,4;v(\bm{x}),p(\bm{x}),q(\bm{x}),r(\bm{x})\in\mathbb{R}[\bm{x}]).

The semidefinite program for solving (12) is formulated below:

where o​b​j=(1α​v​(0,𝒙0)+βα2)​(eα​T−1)−βα​TT−2​MT​if​α≠0obj=\frac{(\frac{1}{\alpha}v(0,\bm{x}_{0})+\frac{\beta}{\alpha^{2}})(e^{\alpha T}-1)-\frac{\beta}{\alpha}T}{T}-\frac{2M}{T}~\text{if}~\alpha\neq 0 and o​b​j=v​(0,𝒙0)+12​β​T−2​MT​if​α=0obj=v(0,\bm{x}_{0})+\frac{1}{2}\beta T-\frac{2M}{T}~\text{if}~\alpha=0, and α\alpha is a user-specified value. The above semi-definite program is minimizing the objective via searching (β;si(t,𝒙)∈∑[t,𝒙],i=0,…,9;v(t,𝒙),w(t,𝒙),p(t,𝒙),q(t,𝒙),r(t,𝒙),l(t,𝒙)∈ℝ[t,𝒙](\beta;s_{i}(t,\bm{x})\in\sum[t,\bm{x}],i=0,\ldots,9;v(t,\bm{x}),w(t,\bm{x}),p(t,\bm{x}),q(t,\bm{x}),r(t,\bm{x}),l(t,\bm{x})\in\mathbb{R}[t,\bm{x}].

The semidefinite program for solving (13) is formulated below:

where o​b​j=(1α​v​(0,𝒙0)+βα2)​(eα​T−1)−βα​TT−2​MT​if​α≠0obj=\frac{(\frac{1}{\alpha}v(0,\bm{x}_{0})+\frac{\beta}{\alpha^{2}})(e^{\alpha T}-1)-\frac{\beta}{\alpha}T}{T}-\frac{2M}{T}~\text{if}~\alpha\neq 0 and o​b​j=v​(0,𝒙0)+12​β​T−2​MT​if​α=0obj=v(0,\bm{x}_{0})+\frac{1}{2}\beta T-\frac{2M}{T}~\text{if}~\alpha=0, and α\alpha is a user-specified value. The above semi-definite program is minimizing the objective via searching (β;si(𝒙)∈∑[𝒙],i=0,…,3;v(𝒙),w(𝒙),p(𝒙),q(𝒙),r(𝒙),l(𝒙)∈ℝ[𝒙](\beta;s_{i}(\bm{x})\in\sum[\bm{x}],i=0,\ldots,3;v(\bm{x}),w(\bm{x}),p(\bm{x}),q(\bm{x}),r(\bm{x}),l(\bm{x})\in\mathbb{R}[\bm{x}].

The semidefinite program for solving (20) is formulated below:

where o​b​j=eα​T​v​(0,𝒙0)+βα​(eα​T−1)​if​α≠0obj=e^{\alpha T}v(0,\bm{x}_{0})+\frac{\beta}{\alpha}(e^{\alpha T}-1)~\text{if}~\alpha\neq 0 and o​b​j=v​(0,𝒙0)+β​T​if​α=0obj=v(0,\bm{x}_{0})+\beta T~\text{if}~\alpha=0, and α\alpha is a user-specified value. The above semi-definite program is minimizing the objective via searching (β;si(t,𝒙)∈∑[t,𝒙],i=0,…,2;sj(𝒙)∈∑[𝒙],j=3,…,5;v(t,𝒙),p(t,𝒙)∈ℝ[t,𝒙](\beta;s_{i}(t,\bm{x})\in\sum[t,\bm{x}],i=0,\ldots,2;s_{j}(\bm{x})\in\sum[\bm{x}],j=3,\ldots,5;v(t,\bm{x}),p(t,\bm{x})\in\mathbb{R}[t,\bm{x}].

The semidefinite program for solving (21) is formulated below:

where o​b​j=eα​T​v​(𝒙0)+βα​(eα​T−1)​if​α≠0obj=e^{\alpha T}v(\bm{x}_{0})+\frac{\beta}{\alpha}(e^{\alpha T}-1)~\text{if}~\alpha\neq 0 and o​b​j=v​(𝒙0)+β​T​if​α=0obj=v(\bm{x}_{0})+\beta T~\text{if}~\alpha=0, and α\alpha is a user-specified value. The above semi-definite program is minimizing the objective via searching (β;si(𝒙)∈∑[𝒙],j=0,…,3;v(𝒙),p(𝒙)∈ℝ[t,𝒙](\beta;s_{i}(\bm{x})\in\sum[\bm{x}],j=0,\ldots,3;v(\bm{x}),p(\bm{x})\in\mathbb{R}[t,\bm{x}].

The semidefinite program for solving (24) is formulated below:

where o​b​j=−(eα​T​v​(0,𝒙0)+βα​(eα​T−1))​if​α≠0obj=-(e^{\alpha T}v(0,\bm{x}_{0})+\frac{\beta}{\alpha}(e^{\alpha T}-1))~\text{if}~\alpha\neq 0 and o​b​j=−(v​(0,𝒙0)+β​T)​if​α=0obj=-(v(0,\bm{x}_{0})+\beta T)~\text{if}~\alpha=0, and α\alpha is a user-specified value. The above semi-definite program is minimizing the objective via searching (β;si(t,𝒙)∈∑[t,𝒙],i=0,…,2;sj(𝒙)∈∑[𝒙],j=3,…,5;v(t,𝒙),p(t,𝒙)∈ℝ[t,𝒙](\beta;s_{i}(t,\bm{x})\in\sum[t,\bm{x}],i=0,\ldots,2;s_{j}(\bm{x})\in\sum[\bm{x}],j=3,\ldots,5;v(t,\bm{x}),p(t,\bm{x})\in\mathbb{R}[t,\bm{x}].

6.2 Connections to Deterministic Reach-avoid Problems

This appendix explores the connections between the stochastic barrier functions proposed in the main text and their counterparts in deterministic systems. When the diffusion term 𝝈​(⋅)\bm{\sigma}(\cdot) is identically zero, the stochastic system (1) reduces to an ordinary differential equation. The following remarks detail how the barrier conditions simplify and can be used to provide guarantees for deterministic reach-avoid problems.

6.3 Explanation on the Ineffectiveness of Condition (25)

Condition (25) is useless in determining lower bounds of the probability ℙ𝒙0T\mathbb{P}_{\bm{x}_{0}}^{T}. We give a brief explanation here. Firstly, we observe that α\alpha should not be zero. If α=0\alpha=0, we have ℙ𝒙0T≥v​(𝒙)+β​T\mathbb{P}_{\bm{x}_{0}}^{T}\geq v(\bm{x})+\beta T for 𝒙∈𝒳∖𝒳s\bm{x}\in\mathcal{X}\setminus\mathcal{X}_{s}. Further, 0≥α​v​(𝒙)+β,∀𝒙∈∂𝒳0\geq\alpha v(\bm{x})+\beta,\forall\bm{x}\in\partial\mathcal{X} implies β≤0\beta\leq 0. Moreover, since v​(𝒙)≤0v(\bm{x})\leq 0 over 𝒙∈𝒳¯∖𝒳s\bm{x}\in\overline{\mathcal{X}}\setminus\mathcal{X}_{s}, we conclude thatv​(𝒙)+β​T≤0v(\bm{x})+\beta T\leq 0 is a useless lower bound. Secondly, if α<0\alpha<0, since v​(𝒙)≤0v(\bm{x})\leq 0 over 𝒙∈𝒳¯∖𝒳s\bm{x}\in\overline{\mathcal{X}}\setminus\mathcal{X}_{s} and 0≥α​v​(𝒙)+β,∀𝒙∈∂𝒳0\geq\alpha v(\bm{x})+\beta,\forall\bm{x}\in\partial\mathcal{X}, we have β≤0\beta\leq 0. Thus, eα​T​v​(𝒙)+βα​(eα​T−1)≤0e^{\alpha T}v(\bm{x})+\frac{\beta}{\alpha}(e^{\alpha T}-1)\leq 0 is a useless lower bound of ℙ𝒙T\mathbb{P}_{\bm{x}}^{T} for 𝒙∈𝒳∖𝒳s\bm{x}\in\mathcal{X}\setminus\mathcal{X}_{s}. Thirdly, if α>0\alpha>0 and β≤0\beta\leq 0, we have eα​T​v​(𝒙0)+βα​(eα​T−1)≤0e^{\alpha T}v(\bm{x}_{0})+\frac{\beta}{\alpha}(e^{\alpha T}-1)\leq 0 is useless lower bound of ℙ𝒙T\mathbb{P}_{\bm{x}}^{T} for 𝒙∈𝒳∖𝒳s\bm{x}\in\mathcal{X}\setminus\mathcal{X}_{s}. Fourthly, if α>0\alpha>0 and β>0\beta>0, we have eα​T​v​(𝒙0)+βα​(eα​T−1)=eα​T​(v​(𝒙0)+βα)−βα≤0e^{\alpha T}v(\bm{x}_{0})+\frac{\beta}{\alpha}(e^{\alpha T}-1)=e^{\alpha T}(v(\bm{x}_{0})+\frac{\beta}{\alpha})-\frac{\beta}{\alpha}\leq 0 is useless lower bound of ℙ𝒙T\mathbb{P}_{\bm{x}}^{T} for 𝒙∈{𝒳∖𝒳s∣v​(𝒙)+βα≤0}\bm{x}\in\{\mathcal{X}\setminus\mathcal{X}_{s}\mid v(\bm{x})+\frac{\beta}{\alpha}\leq 0\}. Lastly, we show that 𝒙∈{𝒳∖𝒳s∣v​(𝒙)+βα>0}=∅\bm{x}\in\{\mathcal{X}\setminus\mathcal{X}_{s}\mid v(\bm{x})+\frac{\beta}{\alpha}>0\}=\emptyset, where α>0\alpha>0 and β>0\beta>0. Assume there exists 𝒙0∈{𝒳∖𝒳s∣v​(𝒙)+βα>0}\bm{x}_{0}\in\{\mathcal{X}\setminus\mathcal{X}_{s}\mid v(\bm{x})+\frac{\beta}{\alpha}>0\}, where α>0\alpha>0 and β>0\beta>0. Following the proof of Theorem 4, we can obtain

Also, since limT→+∞eα​T​(v​(𝒙0)+βα)−βα=+∞\lim_{T\rightarrow+\infty}e^{\alpha T}(v(\bm{x}_{0})+\frac{\beta}{\alpha})-\frac{\beta}{\alpha}=+\infty, limT→+∞𝔼​[v​(𝑿~𝒙0𝒘​(T))]=+∞\lim_{T\rightarrow+\infty}\mathbb{E}[v(\widetilde{\bm{X}}_{\bm{x}_{0}}^{\bm{w}}(T))]=+\infty holds, which contradicts v​(𝒙)≤1𝒳s​(𝒙),∀𝒙∈𝒳¯v(\bm{x})\leq 1_{\mathcal{X}_{s}}(\bm{x}),\forall\bm{x}\in\overline{\mathcal{X}}. Therefore, 𝒙∈{𝒳∖𝒳s∣v​(𝒙)+βα>0}=∅\bm{x}\in\{\mathcal{X}\setminus\mathcal{X}_{s}\mid v(\bm{x})+\frac{\beta}{\alpha}>0\}=\emptyset, where α>0\alpha>0 and β>0\beta>0. In summary, condition (25) is useless in determining lower bounds of the probability ℙ𝒙0T\mathbb{P}_{\bm{x}_{0}}^{T}.