Sensitivity Analysis for Markov Decision Process Congestion Games (original) (raw)

Solution-based congestion games

In this paper we develop the theory of potential of cooperative games for semivalues, characterize congestion models that are defined by semivalues, and suggest an application of these results to combinatorial auctions, which may explain the success of the Iowa electronic market. and they have been analyzed by several researchers from various additional fields, in particular computer science 2 , communication networks 3 , and economics/game theory. 4

On the Inefficiency Ratio of Stable Equilibria in Congestion Games

Lecture Notes in Computer Science, 2009

Price of anarchy and price of stability are the primary notions for measuring the efficiency (i.e. the social welfare) of the outcome of a game. Both of these notions focus on extreme cases: one is defined as the inefficiency ratio of the worst-case equilibrium and the other as the best one. Therefore, studying these notions often results in discovering equilibria that are not necessarily the most likely outcomes of the dynamics of selfish and non-coordinating agents. The current paper studies the inefficiency of the equilibria that are most stable in the presence of noise. In particular, we study two variations of non-cooperative games: atomic congestion games and selfish load balancing. The noisy best-response dynamics in these games keeps the joint action profile around a particular set of equilibria that minimize the potential function. The inefficiency ratio in the neighborhood of these "stable" equilibria is much better than the price of anarchy. Furthermore, the dynamics reaches these equilibria in polynomial time. Our observations show that in the game environments where a small noise is present, the system as a whole works better than what a pessimist may predict. They also suggest that in congestion games, introducing a small noise in the payoff of the agents may improve the social welfare.

Congestion Models and Weighted Bayesian Potential Games

1997

Games associated to congestion situations a la Rosenthal (1973) have pure Nash equilibria. This result implicitly relies on the existence of a potential function. In this paper we will provide a characterization of potential games in terms of coordination games and dummy games. Secondly, w e extend Rosenthal's congestion model to an incomplete information setting, and show that the related Bayesian games are potential games and therefore have pure Bayesian equilibria.

On the inefficiency of equilibria in congestion games

2005

We present a short geometric proof of the price of anarchy and price of stability results that have recently been established in a series of papers on selfish routing. This novel proof also facilitates two types of new results: On the one hand, we give pseudoapproximation results that depend on the class of allowable cost functions. On the other hand, we offer improved bounds on the inefficiency of Nash equilibria for situations in which the equilibrium travel times are within reasonable limits of the free-flow travel times, a scenario that captures empirical observations in vehicular traffic networks. Our results actually hold in the more general context of congestion games, which provide the framework in which we describe this work.

Bounding the inefficiency of equilibria in nonatomic congestion games

Games and Economic Behavior, 2004

Equilibria in noncooperative games are typically inefficient, as illustrated by the Prisoner's Dilemma. In this paper, we quantify this inefficiency by comparing the payoffs of equilibria to the payoffs of a "best possible" outcome. We study a nonatomic version of the congestion games defined by Rosenthal , and identify games in which equilibria are approximately optimal in the sense that no other outcome achieves a significantly larger total payoff to the players-games in which optimization by individuals approximately optimizes the social good, in spite of the lack of coordination between players. Our results extend previous work on traffic routing games . * We thank Lou Billera for encouraging us to explore generalizations of our previous work on traffic routing, and Robert Rosenthal for introducing us to his congestion games. The first author thanks Amir Ronen for comments on a previous paper [16] that facilitated a generalization of the results therein to the nonatomic congestion games of this paper.

Online Constraint Satisfaction via Tolls in MDP Congestion Games

ArXiv, 2019

We consider the toll design problem that arise for a game designer of a congested stochastic network when the decision makers are myopically updating their strategies based on current system state. If both the designer and the decision makers are adaptively searching for an optimal strategy, it is not clear how and if both parties can attain their optimal strategies. We formulate the toll synthesis problem of inducing an equilibrium distribution that satisfies a given set of design specifications as a bilevel optimization problem, in which the lower level consists of decision makers playing a Markov decision process (MDP) congestion game. In addition to showing the existence of an optimal tolling threshold, we formulate learning algorithms that can be employed by both the game designer and the players to jointly determine the optimal toll and induced equilibrium, respectively, and analyze its convergence properties.

Modeling bounded rationality in congestion games with the quantal response equilibrium

This paper investigates the boundedly rational route choice problem with the framework of quantal response equilibrium in which users are noisy optimizers to make route choice decisions. In the congestion game, we establish the boundedly rational route choice model together with a numerical example, and then extend the model with heterogeneous types of users. state, no user can further improve her or his utility by unilaterally changing routes. By relaxing some of the behavioral restrictions implied in a strict deterministic disutility minimization rule, developed a stochastic user equilibrium (SUE) model that considers the travellers' imperfect perceptions of travel times. In this model, the travel time of a link is treated as a random variable which follows some known probability distribution. Gumbel ) and normal distributions are two commonly used ones, which result in the well-known logit-based and probit-based route choice models, respectively. The SUE is achieved when users can no longer change their perceived utility. Existence and uniqueness of UE or SUE in general networks have been well investigated in the literature, including the solution methods for obtaining these two states, see , for more details.

Atomic Congestion Games: Fast, Myopic and Concurrent

Theory of Computing Systems / Mathematical Systems Theory, 2008

We study here the effect of concurrent greedy moves of players in atomic congestion games where n selfish agents (players) wish to select a resource each (out of m resources) so that her selfish delay there is not much. The problem of “maintaining” global progress while allowing concurrent play is exactly what is examined and answered here. We examine two orthogonal settings : (i) A game where the players decide their moves without global information, each acting “freely” by sampling resources randomly and locally deciding to migrate (if the new resource is better) via a random experiment. Here, the resources can have quite arbitrary latency that is load dependent. (ii) An “organised” setting where the players are pre-partitioned into selfish groups (coalitions) and where each coalition does an improving coalitional move. Our work considers concurrent selfish play for arbitrary latencies for the first time. Also, this is the first time where fast coalitional convergence to an approximate equilibrium is shown.

Risk-Averse Equilibria for Vehicle Navigation in Stochastic Congestion Games

IEEE Transactions on Intelligent Transportation Systems

The fast-growing market of autonomous vehicles, unmanned aerial vehicles, and fleets in general necessitates the design of smart and automatic navigation systems considering the stochastic latency along different paths in the traffic network. The longstanding shortest path problem in a deterministic network, whose counterpart in a congestion game setting is Wardrop equilibrium, has been studied extensively, but it is well known that finding the notion of an optimal path is challenging in a traffic network with stochastic arc delays. In this work, we propose three classes of risk-averse equilibria for an atomic stochastic congestion game in its general form where the arc delay distributions are load dependent and not necessarily independent of each other. The three classes are risk-averse equilibrium (RAE), mean-variance equilibrium (MVE), and conditional value at risk level α equilibrium (CVaR α E) whose notions of risk-averse best responses are based on maximizing the probability of taking the shortest path, minimizing a linear combination of mean and variance of path delay, and minimizing the expected delay at a specified risky quantile of the delay distributions, respectively. We prove that for any finite stochastic atomic congestion game, the risk-averse, mean-variance, and CVaR α equilibria exist. We show that for risk-averse travelers, the Braess paradox may not occur to the extent presented originally since players do not necessarily travel along the shortest path in expectation, but they take the uncertainty of travel time into consideration as well. We show through some examples that the price of anarchy can be improved when players are risk-averse and travel according to one of the three classes of risk-averse equilibria rather than the Wardrop equilibrium.

Multiplicative updates outperform generic no-regret learning in congestion games

Proceedings of the 41st annual ACM symposium on Symposium on theory of computing - STOC '09, 2009

We study the outcome of natural learning algorithms in atomic congestion games. Atomic congestion games have a wide variety of equilibria often with vastly differing social costs. We show that in almost all such games, the wellknown multiplicative-weights learning algorithm results in convergence to pure equilibria. Our results show that natural learning behavior can avoid bad outcomes predicted by the price of anarchy in atomic congestion games such as the load-balancing game introduced by Koutsoupias and Papadimitriou, which has super-constant price of anarchy and has correlated equilibria that are exponentially worse than any mixed Nash equilibrium.