A Methodology for Modeling and Representing Expert Knowledge that Supports Teaching-Based Intelligent Agent Development (original) (raw)
Related papers
Ai Magazine, 2001
In his invited talk at the 1993 National Conference on Artificial Intelligence, Edward Feigenbaum compared the technology of a knowledge based computer system with a tiger in a cage. Rarely does a technology arise that offers such a wide range of important benefits. Yet, this technology is still very far from achieving its potential. This tiger is in a cage and to free it the Artificial Intelligence research community must understand and remove the bars of the cage.
Disciple-COA: From Agent Programming to Agent Teaching
Proceedings of the Seventeenth International Conference on Machine Learning, 2000
This paper presents Disciple-COA, the most recent learning agent shell developed in the Disciple framework that aims at changing the way an intelligent agent is built: from "being programmed" by a knowledge engineer, to "being taught" by a domain expert. Disciple-COA can collaborate with the expert to develop its knowledge base consisting of a frame-based ontology that defines the terms from the application domain, and a set of plausible version space rules expressed with these terms. Its central component is a plausible reasoner that can distinguish between four types of problem solving situations: routine, innovative, inventive and creative. This ability guides the interactions with the expert during which the agent learns general rules from specific examples, by integrating a wide range of knowledge acquisition and machine learning strategies, including apprenticeship learning, empirical inductive learning from examples and explanations, and analogical learning. Disciple-COA was developed in the DARPA's High Performance Knowledge Bases program to solve the challenge problem of critiquing military courses of action that were developed as hasty candidate plans for ground combat operations. We present the course of action challenge problem, the process of teaching Disciple-COA to solve it, and the results of DARPA's evaluation in which Disciple-COA demonstrated the best knowledge acquisition rate and problem solving performance. We also present a separate knowledge acquisition experiment conducted at the Battle Command Battle Lab where experts with no prior knowledge engineering experience succeeded to rapidly teach Disciple-COA to correctly critique courses of action.
Rapid Development of a High Performance Knowledge Base for Course of Action Critiquing
2000
This paper presents a practical learning-based methodology and agent shell for building knowledge bases and knowledge-based agents, and their innovative application to the development of a critiquing agent for military courses of action, a challenge problem set by DARPA's High Performance Knowledge Bases program. The agent shell consists of an integrated set of knowledge acquisition, learning and problem solving modules for a generic knowledge base structured into two main components: an ontology that defines the concepts from a specific application domain, and a set of task reduction rules expressed with these concepts. The rapid development of the COA critiquing agent was done by importing an initial ontology from CYC and by teaching the agent to perform its tasks in a way that resembles how an expert would teach a human apprentice when solving problems in cooperation. The methodology, the agent shell, and the developed critiquer were evaluated in several intensive studies, and demonstrated very good results.
THE DISCIPLE-RKF LEARNING AND REASONING AGENT
Computational Intelligence, 2005
Over the years we have developed the Disciple theory, methodology, and family of tools for building knowledge-based agents. This approach consists in developing an agent shell that can be taught directly by a subject matter expert, in a way that resembles how the expert would teach a human apprentice when solving problems in cooperation. This paper presents the most recent version of the Disciple approach and its implementation in the Disciple-RKF system. Disciple-RKF is based on methods for mixed-initiative problem solving, where the expert solves the more creative problems and the agent solves the more routine ones, integrated teaching and learning, where the agent helps the expert to teach it, by asking relevant questions, and the expert helps the agent to learn, by providing examples, hints and explanations, and multistrategy learning, where the agent integrates multiple learning strategies, such as learning from examples, learning from explanations, and learning by analogy, to learn from the expert how to solves problems.
Improving Agent Learning through Rule Analysis
2005
This paper addresses the problem of improving the process by which an agent learns problem solving rules from a subject matter expert. It presents two complementary rule analysis methods that discover when a rule was learned from incomplete explanations of an example, guiding the expert to provide additional explanations. One method performs a structural analysis of the learned rule, while the other method analyzes the possible rule instantiations. Both methods have been implemented in the Disciple-RKF learning agent and have been tested both in an automatic framework and during two knowledge acquisition experiments performed with subject matter experts at the US Army War College. Disciple-RKF uses task-reduction as its main problem solving paradigm. In this paradigm, a problem solving task is successively reduced to simpler tasks, the solutions of the simplest tasks are found, and these solutions are successively composed into the solution of the initial task. The knowledge base of the agent is structured into an object ontology that represents the objects from an application domain, and a set of task reduction rules and solution composition rules expressed with these objects. Disciple-RKF is a general problem solving and learning agent with no specific knowledge in its knowledge base. To develop an agent for a specific application domain, one needs to
Mixed-initiative assistant for modeling expert’s reasoning
2005
This paper presents a mixed-initiative assistant that helps a subject matter expert to express the way she solves problems in the task reduction paradigm. It guides the expert to follow a predefined modeling methodology, supports the expert to express her reasoning by using natural language with references to the objects from the agent's ontology, and helps her in the process of specifying solutions to new problems by analogy with previously solved problems. The assistant, which is integrated into the Disciple system for agents development, has been successfully evaluated by subject matter experts at the US Army War College.
Automating the Acquisition of Tactical Knowledge for Military Missions
The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology, 2005
It is widely accepted that acquisition of the knowledge behind military tactics has been one limiting factor in the development of computer generated forces (CGF) for training simulations. This has been addressed by several researchers with varying degrees of success. A system capable of building a knowledge base directly from a dialogue with a subjectmatter expert (SME) could significantly reduce the human effort involved in capturing the knowledge and representing it directly in the modeling language. Because of its highly modular and hierarchical nature, the context-based reasoning (CxBR) modeling paradigm lends itself very well to facilitating the knowledge acquisition process for tactical behaviors. This paper describes an investigation into using CxBR as the foundation for a system that creates a (partial) model of tactical behavior through an interactive process with an SME. Through a sequence of queries from the system, the SME is progressively asked to provide details about the contexts that compose the context-based model of the expert's tactical know-how. A prototype was built and evaluated. A comparison to the effort taken to manually develop a knowledge base is reported. We use the simulation of a non-trivial maritime military confrontation as the benchmark for the comparisons.
Designing and Implementing Intelligent Tutoring Instruction for Tactical Action Officers
The Tactical Action Officer on board a U.S. Navy Cruiser, Destroyer, or Frigate is responsible for the operation of the entire watch team manning the ship's command center. Responsibilities include tactical decision making, console operation, communications, and oversight of a variety of watchstander responsibilities in air, surface, and subsurface warfare areas. Stottler Henke, in concert with Northrop Grumman, has developed the PORTS TAO ITS, an Intelligent Tutoring System (ITS) for the instruction of Tactical Action Officers (TAOs) in training at the Surface Warfare Officers School (SWOS) using the PC-based Open-architecture Reconfigurable Training System (PORTS) as its basis. This paper describes the instructional philosophy of the PORTS TAO ITS, resulting from close collaboration with SWOS instructors and Northrop Grumman's domain experts. The goal of the ITS is to train the student in 'command by negation,' in which watchstanders perform their duties autonomously, while the TAO supervises, intervening in order to correct mistakes and rectify omissions. The TAO must know what responsibilities belong to each watchstander, how and in what circumstances those duties are performed, and how to communicate with watchstanders to request information, acknowledge reports, and order appropriate actions. The ITS is designed to instill and assess mastery of these TAO abilities over the course of a series of exercises which present increasingly difficult problems. These include intentional mistakes of omission or commission by automated watch team role players. When TAO actions are expected, such as when these intentional mistakes are made, the ITS provides hints, prompts, and feedback to the student, which are also summarized at the end of each exercise with a detailed debrief. These interventions are sensitive to real-time changes in the student's mastery of a wide variety of principles, which are continually assessed. There were several challenges and lessons learned from the implementation of this ITS and the related government acquisition process. These are also detailed in this paper. ABOUT THE AUTHORS Richard Stottler co-founded Stottler Henke Associates, Inc., an artificial intelligence consulting firm in San Mateo, California, in 1988 and has been the president of the company since then. He has been the principal investigator on a large number of tactical decision-making intelligent tutoring system projects conducted by Stottler Henke including projects for the Navy, Army, Air Force and Marine Corps. Currently he is working on the PORTS TAO ITS for the US Navy, and a Littoral Combat Ship project. He has a Masters degree in Computer Science from Stanford University. Alex Davis is an Artificial Intelligence Researcher at Stottler Henke Associates, Inc. He received his M.S. from the State University of New York at Buffalo in Computer Science. He has served as lead knowledge and software engineer for a variety of projects related to artificial intelligence, including simulations, behavior modeling for automated agents, and intelligent tutoring systems. Those projects include the PORTS intelligent tutoring system for Navy tactical action officers, a gaming testbed for the exploration of advanced C2 concepts, and a decision aid that incorporates behavior of adversarial forces in an unfamiliar cultural climate.
A Tutoring Based Approach to the Development of Intelligent Agents
International Series in Intelligent Technologies, 2000
This chapter introduces the concept of intelligent agent, analyses some of the issues and trends in developing them and presents a specific agent development approach. The presented approach, called Disciple, relies on importing ontologies from existing repositories of knowledge, and on teaching the agent how to perform various tasks, in a way that resembles how an expert would teach a human apprentice when solving problems in cooperation.
Evaluating expert-authored rules for military reasoning
Proceedings of the 2nd …, 2003
Eliciting complex logical rules directly from logic-naïve subject matter experts (SMEs) is a challenging knowledge capture task. We describe a large-scale experiment to evaluate tools designed to produce SME-authored rule bases. We assess the quality of the rule bases with respect to the: 1) performance on the addressed functional task (military course of action (COA) critiquing); and 2) intrinsic knowledge representation quality. In the course of this assessment, we note both strengths and weaknesses in the state of the art, and accordingly suggest some foci for future development in this important technology area.