An analysis of stochastic shortest path problems (original) (raw)

A note on the stochastic shortest-route problem

Naval Research Logistics Quarterly

This paper develops an algorithm for a "shortest route" network problem in which it is desired to find the path which yields the shortest expected distance through the network. It is assumed that if a particular arc is chosen, then there is a finite probability that an adjacent arc will be traversed instead. Backward induction is used and appropriate recursion formulae are developed. A numerical example is provided.

Stochastic shortest path with unlimited hops

Information Processing Letters, 2009

We present new results for the Stochastic Shortest Path problem when an unlimited number of hops may be used. Nodes and links in the network may be congested or uncongested, and their states change over time. The goal is to adaptively choose the next node to visit such that the expected cost to the destination is minimized. Since the state of a node may change, having the option to revisit a node can improve the expected cost. Therefore, the option to use an unbounded number of hops may be required to achieve the minimum expected cost. We show that when revisits are prohibited, the optimal routing problem is np-hard. We also prove properties about networks for which continual improvement may occur. We study the related routing problem which asks whether it is possible to determine the optimal next node based on the current node and state, when an unlimited number of hops is allowed. We show that as the number of hops increases, this problem may not converge to a solution.

Shortest paths in stochastic networks

Proceedings. 2004 12th IEEE International Conference on Networks (ICON 2004) (IEEE Cat. No.04EX955), 2004

This paper discusses the sensitivity of network flows to uncertain link state information for various routing protocols. We show that the choice of probability distribution for the link metrics for a given network can have markedly different effects on the probabilities of path selection. Exact results are obtained for these probabilities but their computation is NP-hard. We provide simulation results for three networks to illustrate the sensitivity of shortest paths to different link metric distributions. We provide results for mean path costs and the k-shortest path algorithm as a comparison.

The stochastic quickest path problem via minimal paths

2010

The quickest path problem, a version of the shortest path problem, is to find a single quickest path that sends a given amount of data from the source to the sink with minimum transmission time. More specifically, the capacity of each arc in a network is assumed to be deterministic. However, in many real-life networks, such as computer systems, telecommunication systems, etc., the capacity of each arc is stochastic due to failure, maintenance, etc. Such a network is named a stochastic-flow network.

The online loop-free stochastic shortest-path problem

COLT-10, 2010

We consider a stochastic extension of the loop-free shortest path problem with adversarial rewards. In this episodic Markov decision problem an agent traverses through an acyclic graph with random transitions: at each step of an episode the agent chooses an action, receives some reward, and arrives at a random next state, where the reward and the distribution of the next state depend on the actual state and the chosen action. We consider the bandit situation when only the reward of the just visited state-action pair is revealed to the agent. For this problem we develop algorithms that perform asymptotically as well as the best stationary policy in hindsight. Assuming that all states are reachable with probability α > 0 under all policies, we give an algorithm and prove that its regret is O(L 2 T |A|/α), where T is the number of episodes, A denotes the (finite) set of actions, and L is the length of the longest path in the graph. Variants of the algorithm are given that improve the dependence on the transition probabilities under specific conditions. The results are also extended to variations of the problem, including the case when the agent competes with time varying policies.

Shortest paths in stochastic networks with correlated link costs

Computers & Mathematics with Applications, 2005

The objective is to minimize expected travel time from any origin to a specific destination in a congestible network with correlated link costs. Each link is assumed to be in one of two possible conditions. Conditional probability density functions for link travel times are assumed known for each condition. Conditions over the traversed links are taken into account for determining the optimal routing strategy for the remaining trip. This problem is treated as a multi-stage adaptive feedback control process. Each stage is described by the physical state (the location of the current decision point) and the information state (the service level of the previously traversed links). Proof of existence and uniqueness of the solution to the basic dynamic programming equations and a solution procedure are provided.

On Shortest Path Problems with “Non-Markovian” Link Contribution to Path Lengths

Lecture Notes in Computer Science, 2000

In this paper we introduce a new class of shortest path problems, where the contribution of a link to the path length computation depends not only on the weight of that link but also on the weights of the links already traversed. This class of problems may be viewed as "non-Markovian". We consider a specific problem that belong to this class, which is encountered in the multimedia data transmission domain. We consider this problem under different conditions and develop algorithms. The shortest path problem in multimedia data transmission environment can be solved in O(n 2) or O(n 3) computational time.

Optimal paths in graphs with stochastic or multidimensional weights

Abst ract This paper formulates a stochastic and a multidimensional oplimal path problem, each as an extension of the shortest path problem. Conditions when existing shortest path methods apply are noted. In each problem instance, a utility function defines preference among candidate paths.

Optimal Paths in Probabilistic Networks

Journal of Mathematical Sciences, 2004

The random graph theory was initially proposed by P. Erdős and A. Rényi in the 1950s-1960s. More recently, B. Bollobás presented the first systematic and extensive treatment of results in the theory of random graphs. Associating to each edge of a random graph a real random variable, we obtain a probabilistic network. The determination of an optimal path between two nodes in a probabilistic network was first studied by H. Frank in 1969. Since then few theoretical results have become known, even though there is a recognizable applicability of this type of network to real problems, namely, to social and telecommunication networks. The mathematical model proposed in this paper maximizes the expected value of a utility function over a directed random network, where the costs related to the arcs are real random variables following Gaussian distributions. We consider the linear, quadratic, and exponential cases, presenting a theoretical formulation based on multi-criteria models as well as the resulting algorithms and computational tests.

Distributionally robust stochastic shortest path problem

Electronic Notes in Discrete Mathematics, 2013

This paper considers a stochastic version of the shortest path problem, the Distributionally Robust Stochastic Shortest Path Problem(DRSSPP) on directed graphs. In this model, the arc costs are deterministic, while each arc has a random delay. The mean vector and the second-moment matrix of the uncertain data are assumed known, but the exact information of the distribution is unknown. A penalty occurs when the given delay constraint is not satisfied. The objective is to minimize the sum of the path cost and the expected path delay penalty. As it is NP-hard, we approximate the DRSSPP with a semidefinite programming (SDP for short) problem, which is solvable in polynomial time and provides tight lower bounds.