Expected payoff analysis of dynamic mixed strategies in an adversarial domain (original) (raw)

A dynamic mixed strategy for an adversarial model based on OWA weights

Adversarial decision making is aimed at determin-ing optimal decision strategies to deal with an adversarial and adaptive opponent. One defense against this adversary is to make decisions that are intended to confuse him, although our rewards can be diminished. In this contribution, we propose time varying decision strategies for a simple adversarial model. These strategies have been obtained by using linear OWA weights that have influence on the decision made. Several time-varying patterns have been used to build such strategies. We have compared their performance against static strategies. The new strategies tested consistently outperform the optimal static strategy in a variety of situations. This is an encouraging result that confirms these strategies deserve further investigation.

A Game Theoretical Framework for Adversarial Learning

Many data mining applications, ranging from spam filtering to intrusion detection, are faced with active adversaries. In all these applications, initially successful classifiers will degrade easily. This becomes a game between the adversary and the data miner: The adversary modifies its strategy to avoid being detected by the current classifier; the data miner then updates its classifier based on the new threats. In this paper, we investigate the possibility of an equilibrium in this seemingly never ending game, where neither party has an incentive to change. Modifying the classifier causes too many false positives with too little increase in true positives; changes by the adversary decrease the utility of the false negative items that aren't detected. We develop a game theoretic framework where the equilibrium behavior of adversarial learning applications can be analyzed, and provide a solution for finding the equilibrium point. A classifier's equilibrium performance indicates its eventual success or failure. The data miner could then select attributes based on their equilibrium performance, and construct an effective classifier.

Evolutionary design and statistical assessment of strategies in an adversarial domain

IEEE Congress on Evolutionary Computation, 2010

Adversarial decision making is aimed at finding strategies for dealing with an adversary who observes our decisions and tries to learn our behaviour pattern. Departing from a simple mathematical model, the present contribution extends it with strategies that vary along time, and motivates the use of heuristic search procedures to address the problem of finding good strategies within this new search space. Evaluation of this new class of strategies requires running a stochastic simulation so the comparison of strategies should be properly addressed. A new statistics-based technique for comparison of strategies is also proposed and tested in this context when coupled with a Genetic Algorithm. Computational experiments showed that the new strategies are better than previous ones, and that the results obtained with this new comparison technique are encouraging.

A game-theoretic approach to assess adversarial risks

Risk Analysis IX, 2014

In our complex world today almost all critical infrastructures are interdependent and thus vulnerable to many different external and internal risks. To protect them against the greatest risks, a well-functioning risk management process is required to develop appropriate safety and security strategies. There are many wellestablished risk analysis methods existing. They predominantly apply empirical models and statistical data to quantify the risks. Within the realms of natural or aleatory risks this approach is considered suitable and functional. However, it could be a fatal flaw to apply such conventional, history-orientated models in order to assess risks that arise from intelligent adversaries such as terrorists, criminals or competitors. Approaches of classic risk analysis generally describe adversaries' choices as random variables, thus excluding the adversaries' behaviour and ability to adapt to security strategies. One possibility for considering human behaviour when analysing risks is the recourse to game theory. Game theory is the paradigmatic framework for strategic decision-making when two or more rational decision-makers (intelligent adversaries) are involved in cooperative or conflictive decision situations. In our study we propose an approach for combining a classic risk analysis method with a game-theoretic approach. Using a defenderoffender game as a basis, we simulate, exemplary for a terrorist attack against public transport, the behaviour and reactions (to applied security strategies of the defender) of a rational player acting as an adversary. Although risk analysis and game theory are very different methodologies, we show that linking them can significantly improve the quality of forecasts and risk assessments. If the behaviour and reactions of intelligent adversaries need to be considered, our approach contributes to enhance security through improving the allocation of scarce financial resources.

Adversarial risk analysis: An overview

WIREs Computational Statistics

Adversarial risk analysis (ARA) is a relatively new area of research that informs decision-making when facing intelligent opponents and uncertain outcomes. It enables an analyst to express her Bayesian beliefs about an opponent's utilities, capabilities, probabilities and the type of strategic calculation that the opponent is using. Within that framework, the analyst then solves the problem from the perspective of the opponent while placing subjective probability distributions on all unknown quantities. This produces a distribution over the actions of the opponent that permits the analyst to maximize her expected utility. This overview covers conceptual, modeling, computational and applied issues in ARA.

Partial Adversarial Behavior Deception in Security Games

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, 2020

Learning attacker behavior is an important research topic in security games as security agencies are often uncertain about attackers' decision making. Previous work has focused on developing various behavioral models of attackers based on historical attack data. However, a clever attacker can manipulate its attacks to fail such attack-driven learning, leading to ineffective defense strategies. We study attacker behavior deception with three main contributions. First, we propose a new model, named partial behavior deception model, in which there is a deceptive attacker (among multiple attackers) who controls a portion of attacks. Our model captures real-world security scenarios such as wildlife protection in which multiple poachers are present. Second, we introduce a new scalable algorithm, GAMBO, to compute an optimal deception strategy of the deceptive attacker. Our algorithm employs the projected gradient descent and uses the implicit function theorem for the computation of gr...

A Survey of Opponent Modeling in Adversarial Domains

Journal of Artificial Intelligence Research

Opponent modeling is the ability to use prior knowledge and observations in order to predict the behavior of an opponent. This survey presents a comprehensive overview of existing opponent modeling techniques for adversarial domains, many of which must address stochastic, continuous, or concurrent actions, and sparse, partially observable payoff structures. We discuss all the components of opponent modeling systems, including feature extraction, learning algorithms, and strategy abstractions. These discussions lead us to propose a new form of analysis for describing and predicting the evolution of game states over time. We then introduce a new framework that facilitates method comparison, analyze a representative selection of techniques using the proposed framework, and highlight common trends among recently proposed methods. Finally, we list several open problems and discuss future research directions inspired by AI research on opponent modeling and related research in other discip...

Opponent Modeling in Interesting Adversarial Environments

We advance the field of research involving modeling opponents in interesting adversarial environments: environments in which equilibrium strategies are intractable to calculate or undesirable to use. We motivate the need for opponent models in such environments by showing how successful opponent modeling agents can exploit nonequilibrium strategies and strategies using equilibrium approximations. We develop a new, flexible measurement which can be used to quantify how well our model can predict the opponent's behavior independently from the performance of the agent in which it resides. We show how this metric can be used to find areas of model improvement that would otherwise have remained undiscovered and demonstrate the technique for evaluating opponent model quality in the poker domain. We introduce the idea of performance bounds for classes of opponent models, present a method for calculating them, and show how these bounds are a function of only the environment and thus inv...

Automated Planning in Repeated Adversarial Games

2010

Game theory's prescriptive power typically relies on full rationality and/or self-play interactions. In contrast, this work sets aside these fundamental premises and focuses instead on heterogeneous autonomous interactions between two or more agents. Specifically, we introduce a new and concise representation for repeated adversarial (constant-sum) games that highlight the necessary features that enable an automated planing agent to reason about how to score above the game's Nash equilibrium, when facing heterogeneous adversaries. To this end, we present TeamUP, a model-based RL algorithm designed for learning and planning such an abstraction. In essence, it is somewhat similar to R-max with a cleverly engineered reward shaping that treats exploration as an adversarial optimization problem. In practice, it attempts to find an ally with which to tacitly collude (in more than two-player games) and then collaborates on a joint plan of actions that can consistently score a high utility in adversarial repeated games.