John Kolen - Academia.edu (original) (raw)
Papers by John Kolen
The MIT Press eBooks, 1999
... Such architectures deal well with sequential information. ... These expectations dramatically... more ... Such architectures deal well with sequential information. ... These expectations dramatically affect human perception, atten-tion and memory for the complex event sequences found in music (Jones & Boltz, 1989; Palmer & Krumhansl, 1990; Povel & Essens, 1985). ...
Proceedings of the AAAI Conference on Artificial Intelligence
Games have been a major focus of AI since the field formed seventy years ago. Recently, video gam... more Games have been a major focus of AI since the field formed seventy years ago. Recently, video games have replaced chess and go as the current "Mt. Everest Problem." This paper looks beyond the video games themselves to the application of AI techniques within the ecosystems that produce them. Electronic Arts (EA) must deal with AI at scale across many game studios as it develops many AAA games each year, and not a single, AI-based, flagship application. EA has adopted a horizontal scaling strategy in response to this challenge and built a platform for delivering AI artifacts anywhere within EA's software universe. By combining a data warehouse for player history, an Agent Store for capturing processes acquired through machine learning, and a recommendation engine as an action layer, EA has been delivering a wide range of AI solutions throughout the company during the last two years. These solutions, such as dynamic difficulty adjustment, in-game content and activity rec...
Embodiments of systems presented herein may identify users to play a multiplayer video game toget... more Embodiments of systems presented herein may identify users to play a multiplayer video game together using a mapping system and machine learning algorithms to create sets of matchmaking plans for the multiplayer video game that increases player or user retention. Embodiments of systems presented herein can determine a predicted churn rate or a retention rate, conversely, of a user waiting to play a video game if the user is matched with one or more additional users in a multiplayer instance of the video game.
This paper discusses the problem of how to implement many-to-many, or multi-associative, mappings... more This paper discusses the problem of how to implement many-to-many, or multi-associative, mappings within connectionist models. Traditional symbolic approaches wield explicit representation of all alternatives via stored links, or implicitly through enumerative algorithms. Classical pattern association models ignore the issue of generating multiple outputs for a single input pattern, and while recent research on recurrent networks is promising, the field has not clearly focused upon multi-associativity as a goal. In this paper, we define multiassociative memory MM, and several possible variants, and discuss its utility in general cognitive modeling. We extend sequential cascaded networks (Pollack 1987, 1990a) to fit the task, and perform several initial experiments which demonstrate the feasibility of the concept. This paper appears in The Proceedings of the Thirteenth Annual Conference of the Cognitive Science Society. August 7-10, 1991. Multiassociative Memory1
IEEE Transactions on Systems, Man, and Cybernetics, 1991
His current research interests include neural networks, computational geometry, and discovering w... more His current research interests include neural networks, computational geometry, and discovering ways to exploit the computational capabilities of nonlinear dynamical systems.
Proceedings of the …, 1998
An AI Approach to Computer Assisted Tomography. John F. Kolen, David A. Shamma, Thomas Reichherze... more An AI Approach to Computer Assisted Tomography. John F. Kolen, David A. Shamma, Thomas Reichherzer, and Timothy Flueharty. Computer assisted tomography (CAT) systems demand large amounts of time and space. In ...
Many programming languages use short-circuit evaluation of boolean expressions (eg C (Kernighan &... more Many programming languages use short-circuit evaluation of boolean expressions (eg C (Kernighan & Ritchie, 1978)). This process eliminates unnecessary evaluation of parts of an expression by terminating the evaluation when a terminating truth value is discovered. For example, if a evaluates to the expression will be regardless of the truth values of and. We extend this notion to the task of deductive reasoning by introducing a new normal form, the telescope expression. We show that the standard logical operations are closed over ...
Matchmaking connects multiple players to participate in online player-versus-player games. Curren... more Matchmaking connects multiple players to participate in online player-versus-player games. Current matchmaking systems depend on a single core strategy: create fair games at all times. These systems pair similarly skilled players on the assumption that a fair game is best player experience. We will demonstrate, however, that this intuitive assumption sometimes fails and that matchmaking based on fairness is not optimal for engagement. In this paper, we propose an Engagement Optimized Matchmaking (EOMM) framework that maximizes overall player engagement. We prove that equal-skill based matchmaking is a special case of EOMM on a highly simplified assumption that rarely holds in reality. Our simulation on real data from a popular game made by Electronic Arts, Inc. (EA) supports our theoretical results, showing significant improvement in enhancing player engagement compared to existing matchmaking methods.
In back-propagation (Rumelhart et al, 1985) connection weights are used to both compute node acti... more In back-propagation (Rumelhart et al, 1985) connection weights are used to both compute node activations and error gradients for hidden units. Grossberg (1987) has argued that the dual use of the same synaptic connections ("weight transport") constitutes a bidirectional flow of information through synapses, which is biologically implausable. In this paper we formally and empirically demonstrate the feasibility of an architecture equivalent to back-propagation, but without the assumption of weight transport. Through coordinated training with weight decay, a reciprocal layer of weights evolves into a copy of the forward connections and acts as the conduit for backward flowing corrective information. Examination of the networks trained with dual weights suggests that functional synchronization, and not weight synchronization, is crucial to the operation of back-propagation methods. Introduction Back-propagation (Rumelhart et al, 1985) is a popular gradient descent method for ...
this paper, clustering of hidden unit activations, or recurrent network state space, provides inc... more this paper, clustering of hidden unit activations, or recurrent network state space, provides incomplete information regarding the IP state of the network. IP states determine future behavior as well as encapsulate input history. The network's state transformations can exhibit sensitivity to initial conditions and generate disparate futures for state clusters of all sizes. The second part of the paper presents IFS theory and shows how it can explain recurrent network state dynamics. By linking IFSs and recurrent networks, existing constraints on network dynamics independent of network models are now evident. By assuming a finite set of inputs, which is often the case in symbolic domains, one can describe recurrent network models as a finite collection of nonlinear state transformations.The interaction of several transforms produces complex state spaces with recursive structure. The limit behavior of the collection of transformations, and recurrent networks in symbolic applicati...
An aviator may control an aircraft by viewing the world through the windscreen (visual flight) or... more An aviator may control an aircraft by viewing the world through the windscreen (visual flight) or with information from the cockpit instruments (instrument flight), depending upon visibility, meteorological conditions, and the competence of the pilot. It seems intuitively obvious that instrument flight should be far more challenging than visual flight. However, since the same pilot controls the same aircraft through the same air using the same stick-and-rudder input devices, the only difference is the way the same information is presented. Consequently, instrument flight is harder than visual flight only because of the way the instruments display the information. Each instrument displays one flight parameter and it is up to the pilot to convert the numeric value of the displayed parameter into useful information. We think that it is possible to use modern display technologies and computational capabilities to make instrument flight safer, easier, and more efficient, possibly even be...
Many researchers in AI and cognitive science believe that the complexity of a behavioral descript... more Many researchers in AI and cognitive science believe that the complexity of a behavioral description reflects the underlying information processing complexity of the mechanism producing the behavior. This paper explores the foundations of this complexity argument. We first distinguish two types of complexity judgements that can be applied to these descriptions and then argue that neither type can be an intrinsic property of the underlying physical system. In short, we demonstrate how changes in the method of observation can radically alter both the number of apparent states and the apparent generative class of a system's behavioral description. From these examples we conclude that the act of observation can suggest frivolous computational explanations of physical phenomena, up to and including cognition. The Observers' Paradox 3 The daily warmth we experience, my father said, is not transmitted by Sun to Earth but is what Earth does in response to Sun. Measurements, he sai...
Recently, there have been several high-profile achievements of agents learning to play games agai... more Recently, there have been several high-profile achievements of agents learning to play games against humans and beat them. We propose an approach that instead addresses how the player experiences the game, which we consider to be a more challenging problem. In this paper, we present an alternative context for developing AI that plays games. Specifically, we study the problem of creating intelligent game agents in service of the development processes of the game developers that design, build, and operate modern games. We highlight some of the ways in which we think intelligent agents can assist game developers to understand their games, and even to build them. Our main contribution is to propose a learning and planning framework that is uniquely tuned to the environment and needs of modern game engines, developers and players. Our game agent framework takes a few steps towards addressing the unique challenges that game developers face. We discuss some early results from an initial im...
From the many possible perspectives in which an agent may be viewed, behavior-based AI selects ob... more From the many possible perspectives in which an agent may be viewed, behavior-based AI selects observable actions as a particularly useful level of description. Yet behavior is clearly not structure, and anyone using behavior-based constraints to construct an agent still faces many implementa-tional roadblocks. Such obstacles are typically avoided by adopting a finite state automaton (FSA) as a base representation. As a result, potential benefits from alternative formalisms are ignored. To explore these benefits, our work adopts a multi-level view of an agent: behaviors and FSAs are but two of many levels of description. We still focus on behaviors for the expression of design constraints, but we avoid using FSAs as an implementation. Our particular agent, Addam, is comprised of a set of connectionist networks, a substrate which promotes the automatic design of subsumptive systems. Moreover, the implementational choice has important behavioral consequences – some complex behaviors e...
ArXiv, 2019
Recently, there have been several high-profile achievements of agents learning to play games agai... more Recently, there have been several high-profile achievements of agents learning to play games against humans and beat them. We consider an alternative approach that instead addresses game design for a better player experience by training human-like game agents. Specifically, we study the problem of training game agents in service of the development processes of the game developers that design, build, and operate modern games. We highlight some of the ways in which we think intelligent agents can assist game developers to understand their games, and even to build them. Our early results using the proposed agent framework mark a few steps toward addressing the unique challenges that game developers face.
The digital game industry has recently adopted recommendation systems to provide suitable game an... more The digital game industry has recently adopted recommendation systems to provide suitable game and content choices to players. Recommendations in digital games have several unique applications and challenges compared to other well known recommendation system such as those for movies and books. Designers must adopt different architectures and algorithms to overcome these challenges. In this talk, we describe the game recommendation system at Electronic Arts. It leverages heterogeneous player data across many games to provide intelligent recommendations. We discuss three example applications: recommending games for purchase, suitable game map, and game difficulty. Like the movie and book recommendation problem, one application is to recommend the next favorite games for a player. Digital games fall into a wide range of genres such as first player shooting (FPS), sports, and role-playing games (RPG). Games within the same genre however tend to be unique and creative. While the recommen...
The process of playtesting a game is subjective, expensive and incomplete. In this paper, we pres... more The process of playtesting a game is subjective, expensive and incomplete. In this paper, we present a playtesting approach that explores the game space with automated agents and collects data to answer questions posed by the designers. Rather than have agents interacting with an actual game client, this approach recreates the bare bone mechanics of the game as a separate system. Our agent is able to play in minutes what would take testers days of organic gameplay. The analysis of thousands of game simulations exposed imbalances in game actions, identified inconsequential rewards and evaluated the effectiveness of optional strategic choices. Our test case game, The Sims Mobile, was recently released and the findings shown here influenced design changes that resulted in improved player experience.
Proceedings of the 26th International Conference on World Wide Web Companion - WWW '17 Companion
The MIT Press eBooks, 1999
... Such architectures deal well with sequential information. ... These expectations dramatically... more ... Such architectures deal well with sequential information. ... These expectations dramatically affect human perception, atten-tion and memory for the complex event sequences found in music (Jones & Boltz, 1989; Palmer & Krumhansl, 1990; Povel & Essens, 1985). ...
Proceedings of the AAAI Conference on Artificial Intelligence
Games have been a major focus of AI since the field formed seventy years ago. Recently, video gam... more Games have been a major focus of AI since the field formed seventy years ago. Recently, video games have replaced chess and go as the current "Mt. Everest Problem." This paper looks beyond the video games themselves to the application of AI techniques within the ecosystems that produce them. Electronic Arts (EA) must deal with AI at scale across many game studios as it develops many AAA games each year, and not a single, AI-based, flagship application. EA has adopted a horizontal scaling strategy in response to this challenge and built a platform for delivering AI artifacts anywhere within EA's software universe. By combining a data warehouse for player history, an Agent Store for capturing processes acquired through machine learning, and a recommendation engine as an action layer, EA has been delivering a wide range of AI solutions throughout the company during the last two years. These solutions, such as dynamic difficulty adjustment, in-game content and activity rec...
Embodiments of systems presented herein may identify users to play a multiplayer video game toget... more Embodiments of systems presented herein may identify users to play a multiplayer video game together using a mapping system and machine learning algorithms to create sets of matchmaking plans for the multiplayer video game that increases player or user retention. Embodiments of systems presented herein can determine a predicted churn rate or a retention rate, conversely, of a user waiting to play a video game if the user is matched with one or more additional users in a multiplayer instance of the video game.
This paper discusses the problem of how to implement many-to-many, or multi-associative, mappings... more This paper discusses the problem of how to implement many-to-many, or multi-associative, mappings within connectionist models. Traditional symbolic approaches wield explicit representation of all alternatives via stored links, or implicitly through enumerative algorithms. Classical pattern association models ignore the issue of generating multiple outputs for a single input pattern, and while recent research on recurrent networks is promising, the field has not clearly focused upon multi-associativity as a goal. In this paper, we define multiassociative memory MM, and several possible variants, and discuss its utility in general cognitive modeling. We extend sequential cascaded networks (Pollack 1987, 1990a) to fit the task, and perform several initial experiments which demonstrate the feasibility of the concept. This paper appears in The Proceedings of the Thirteenth Annual Conference of the Cognitive Science Society. August 7-10, 1991. Multiassociative Memory1
IEEE Transactions on Systems, Man, and Cybernetics, 1991
His current research interests include neural networks, computational geometry, and discovering w... more His current research interests include neural networks, computational geometry, and discovering ways to exploit the computational capabilities of nonlinear dynamical systems.
Proceedings of the …, 1998
An AI Approach to Computer Assisted Tomography. John F. Kolen, David A. Shamma, Thomas Reichherze... more An AI Approach to Computer Assisted Tomography. John F. Kolen, David A. Shamma, Thomas Reichherzer, and Timothy Flueharty. Computer assisted tomography (CAT) systems demand large amounts of time and space. In ...
Many programming languages use short-circuit evaluation of boolean expressions (eg C (Kernighan &... more Many programming languages use short-circuit evaluation of boolean expressions (eg C (Kernighan & Ritchie, 1978)). This process eliminates unnecessary evaluation of parts of an expression by terminating the evaluation when a terminating truth value is discovered. For example, if a evaluates to the expression will be regardless of the truth values of and. We extend this notion to the task of deductive reasoning by introducing a new normal form, the telescope expression. We show that the standard logical operations are closed over ...
Matchmaking connects multiple players to participate in online player-versus-player games. Curren... more Matchmaking connects multiple players to participate in online player-versus-player games. Current matchmaking systems depend on a single core strategy: create fair games at all times. These systems pair similarly skilled players on the assumption that a fair game is best player experience. We will demonstrate, however, that this intuitive assumption sometimes fails and that matchmaking based on fairness is not optimal for engagement. In this paper, we propose an Engagement Optimized Matchmaking (EOMM) framework that maximizes overall player engagement. We prove that equal-skill based matchmaking is a special case of EOMM on a highly simplified assumption that rarely holds in reality. Our simulation on real data from a popular game made by Electronic Arts, Inc. (EA) supports our theoretical results, showing significant improvement in enhancing player engagement compared to existing matchmaking methods.
In back-propagation (Rumelhart et al, 1985) connection weights are used to both compute node acti... more In back-propagation (Rumelhart et al, 1985) connection weights are used to both compute node activations and error gradients for hidden units. Grossberg (1987) has argued that the dual use of the same synaptic connections ("weight transport") constitutes a bidirectional flow of information through synapses, which is biologically implausable. In this paper we formally and empirically demonstrate the feasibility of an architecture equivalent to back-propagation, but without the assumption of weight transport. Through coordinated training with weight decay, a reciprocal layer of weights evolves into a copy of the forward connections and acts as the conduit for backward flowing corrective information. Examination of the networks trained with dual weights suggests that functional synchronization, and not weight synchronization, is crucial to the operation of back-propagation methods. Introduction Back-propagation (Rumelhart et al, 1985) is a popular gradient descent method for ...
this paper, clustering of hidden unit activations, or recurrent network state space, provides inc... more this paper, clustering of hidden unit activations, or recurrent network state space, provides incomplete information regarding the IP state of the network. IP states determine future behavior as well as encapsulate input history. The network's state transformations can exhibit sensitivity to initial conditions and generate disparate futures for state clusters of all sizes. The second part of the paper presents IFS theory and shows how it can explain recurrent network state dynamics. By linking IFSs and recurrent networks, existing constraints on network dynamics independent of network models are now evident. By assuming a finite set of inputs, which is often the case in symbolic domains, one can describe recurrent network models as a finite collection of nonlinear state transformations.The interaction of several transforms produces complex state spaces with recursive structure. The limit behavior of the collection of transformations, and recurrent networks in symbolic applicati...
An aviator may control an aircraft by viewing the world through the windscreen (visual flight) or... more An aviator may control an aircraft by viewing the world through the windscreen (visual flight) or with information from the cockpit instruments (instrument flight), depending upon visibility, meteorological conditions, and the competence of the pilot. It seems intuitively obvious that instrument flight should be far more challenging than visual flight. However, since the same pilot controls the same aircraft through the same air using the same stick-and-rudder input devices, the only difference is the way the same information is presented. Consequently, instrument flight is harder than visual flight only because of the way the instruments display the information. Each instrument displays one flight parameter and it is up to the pilot to convert the numeric value of the displayed parameter into useful information. We think that it is possible to use modern display technologies and computational capabilities to make instrument flight safer, easier, and more efficient, possibly even be...
Many researchers in AI and cognitive science believe that the complexity of a behavioral descript... more Many researchers in AI and cognitive science believe that the complexity of a behavioral description reflects the underlying information processing complexity of the mechanism producing the behavior. This paper explores the foundations of this complexity argument. We first distinguish two types of complexity judgements that can be applied to these descriptions and then argue that neither type can be an intrinsic property of the underlying physical system. In short, we demonstrate how changes in the method of observation can radically alter both the number of apparent states and the apparent generative class of a system's behavioral description. From these examples we conclude that the act of observation can suggest frivolous computational explanations of physical phenomena, up to and including cognition. The Observers' Paradox 3 The daily warmth we experience, my father said, is not transmitted by Sun to Earth but is what Earth does in response to Sun. Measurements, he sai...
Recently, there have been several high-profile achievements of agents learning to play games agai... more Recently, there have been several high-profile achievements of agents learning to play games against humans and beat them. We propose an approach that instead addresses how the player experiences the game, which we consider to be a more challenging problem. In this paper, we present an alternative context for developing AI that plays games. Specifically, we study the problem of creating intelligent game agents in service of the development processes of the game developers that design, build, and operate modern games. We highlight some of the ways in which we think intelligent agents can assist game developers to understand their games, and even to build them. Our main contribution is to propose a learning and planning framework that is uniquely tuned to the environment and needs of modern game engines, developers and players. Our game agent framework takes a few steps towards addressing the unique challenges that game developers face. We discuss some early results from an initial im...
From the many possible perspectives in which an agent may be viewed, behavior-based AI selects ob... more From the many possible perspectives in which an agent may be viewed, behavior-based AI selects observable actions as a particularly useful level of description. Yet behavior is clearly not structure, and anyone using behavior-based constraints to construct an agent still faces many implementa-tional roadblocks. Such obstacles are typically avoided by adopting a finite state automaton (FSA) as a base representation. As a result, potential benefits from alternative formalisms are ignored. To explore these benefits, our work adopts a multi-level view of an agent: behaviors and FSAs are but two of many levels of description. We still focus on behaviors for the expression of design constraints, but we avoid using FSAs as an implementation. Our particular agent, Addam, is comprised of a set of connectionist networks, a substrate which promotes the automatic design of subsumptive systems. Moreover, the implementational choice has important behavioral consequences – some complex behaviors e...
ArXiv, 2019
Recently, there have been several high-profile achievements of agents learning to play games agai... more Recently, there have been several high-profile achievements of agents learning to play games against humans and beat them. We consider an alternative approach that instead addresses game design for a better player experience by training human-like game agents. Specifically, we study the problem of training game agents in service of the development processes of the game developers that design, build, and operate modern games. We highlight some of the ways in which we think intelligent agents can assist game developers to understand their games, and even to build them. Our early results using the proposed agent framework mark a few steps toward addressing the unique challenges that game developers face.
The digital game industry has recently adopted recommendation systems to provide suitable game an... more The digital game industry has recently adopted recommendation systems to provide suitable game and content choices to players. Recommendations in digital games have several unique applications and challenges compared to other well known recommendation system such as those for movies and books. Designers must adopt different architectures and algorithms to overcome these challenges. In this talk, we describe the game recommendation system at Electronic Arts. It leverages heterogeneous player data across many games to provide intelligent recommendations. We discuss three example applications: recommending games for purchase, suitable game map, and game difficulty. Like the movie and book recommendation problem, one application is to recommend the next favorite games for a player. Digital games fall into a wide range of genres such as first player shooting (FPS), sports, and role-playing games (RPG). Games within the same genre however tend to be unique and creative. While the recommen...
The process of playtesting a game is subjective, expensive and incomplete. In this paper, we pres... more The process of playtesting a game is subjective, expensive and incomplete. In this paper, we present a playtesting approach that explores the game space with automated agents and collects data to answer questions posed by the designers. Rather than have agents interacting with an actual game client, this approach recreates the bare bone mechanics of the game as a separate system. Our agent is able to play in minutes what would take testers days of organic gameplay. The analysis of thousands of game simulations exposed imbalances in game actions, identified inconsequential rewards and evaluated the effectiveness of optional strategic choices. Our test case game, The Sims Mobile, was recently released and the findings shown here influenced design changes that resulted in improved player experience.
Proceedings of the 26th International Conference on World Wide Web Companion - WWW '17 Companion