Pulkit Verma - Academia.edu (original) (raw)

Papers by Pulkit Verma

Research paper thumbnail of Can LLMs Converse Formally? Automatically Assessing LLMs in Translating and Interpreting Formal Specifications

arXiv (Cornell University), Mar 27, 2024

Research paper thumbnail of Data Efficient Paradigms for Personalized Assessment of Black-Box Taskable AI Systems

Proceedings of the ... AAAI Conference on Artificial Intelligence, Mar 24, 2024

The vast diversity of internal designs of taskable black-box AI systems and their nuanced zones o... more The vast diversity of internal designs of taskable black-box AI systems and their nuanced zones of safe functionality make it difficult for a layperson to use them without unintended side effects. My dissertation focuses on developing paradigms that enable a user to assess and understand the limits of an AI system's safe operability. We develop a personalized AI assessment module that lets an AI system execute instruction sequences in simulators and answer queries about these executions. Our results show that such a primitive query-response interface is sufficient to efficiently derive a user-interpretable model of a system's capabilities.

Research paper thumbnail of From Reals to Logic and Back: Inventing Symbolic Vocabularies, Actions and Models for Planning from Raw Data

arXiv (Cornell University), Feb 19, 2024

Hand-crafted, logic-based state and action representations have been widely used to overcome the ... more Hand-crafted, logic-based state and action representations have been widely used to overcome the intractable computational complexity of long-horizon robot planning problems, including task and motion planning problems. However, creating such representations requires experts with strong intuitions and detailed knowledge about the robot and the tasks it may need to accomplish in a given setting. Removing this dependency on human intuition is a highly active research area. This paper presents the first approach for autonomously learning generalizable, logic-based relational representations for abstract states and actions starting from unannotated high-dimensional, real-valued robot trajectories. The learned representations constitute auto-invented PDDL-like domain models. Empirical results in deterministic settings show that powerful abstract representations can be learned from just a handful of robot trajectories; the learned relational representations include but go beyond classical, intuitive notions of high-level actions; and that the learned models allow planning algorithms to scale to tasks that were previously beyond the scope of planning without hand-crafted abstractions.

Research paper thumbnail of Epistemic Exploration for Generalizable Planning and Learning in Non-Stationary Settings

arXiv (Cornell University), Feb 12, 2024

This paper introduces a new approach for continual planning and model learning in relational, non... more This paper introduces a new approach for continual planning and model learning in relational, non-stationary stochastic environments. Such capabilities are essential for the deployment of sequential decision-making systems in the uncertain and constantly evolving real world. Working in such practical settings with unknown (and non-stationary) transition systems and changing tasks, the proposed framework models gaps in the agent's current state of knowledge and uses them to conduct focused, investigative explorations. Data collected using these explorations is used for learning generalizable probabilistic models for solving the current task despite continual changes in the environment dynamics. Empirical evaluations on several non-stationary benchmark domains show that this approach significantly outperforms planning and RL baselines in terms of sample complexity. Theoretical results show that the system exhibits desirable convergence properties when stationarity holds.

Research paper thumbnail of Data Efficient Algorithms and Interpretability Requirements for Personalized Assessment of Taskable AI Systems

The vast diversity of internal designs of taskable black-box AI systems and their nuanced zones o... more The vast diversity of internal designs of taskable black-box AI systems and their nuanced zones of safe functionality make it difficult for a layperson to use them without unintended side effects. The focus of my dissertation is to develop algorithms and requirements of interpretability that would enable a user to assess and understand the limits of an AI system's safe operability. We develop an assessment module that lets an AI system execute high-level instruction sequences in simulators and answer the user queries about its execution of sequences of actions. Our results show that such a primitive query-response capability is sufficient to efficiently derive a user-interpretable model of the system in stationary, fully observable, and deterministic settings.

Research paper thumbnail of A comparative study of resource usage for speaker recognition techniques

Resource usage of a software is an important factor to be taken into consideration while developi... more Resource usage of a software is an important factor to be taken into consideration while developing speaker recognition applications for mobile devices. Sometimes usage parameters are considered as important as accuracy of such systems. In this work, we analyze resource utilization in terms of power consumption, memory and space requirements of three standard speaker recognition techniques, viz. GMM-UBM framework, Joint Factor Analysis and i-vectors. Experiments are performed on the MIT MDSVC corpus using the Energy Measurement Library (EML). It is found that though i-vector approach requires more storage space, it is superior to the other two approaches in terms of memory and power consumption, which are critical factors for evaluating software performance in resource constrained mobile devices.

Research paper thumbnail of Autonomous Capability Assessment of Black-Box Sequential Decision-Making Systems

arXiv (Cornell University), Jun 7, 2023

It is essential for users to understand what their AI systems can and can't do in order to use th... more It is essential for users to understand what their AI systems can and can't do in order to use them safely. However, the problem of enabling users to assess AI systems with sequential decision-making (SDM) capabilities is relatively understudied. This paper presents a new approach for modeling the capabilities of black-box AI systems that can plan and act, along with the possible effects and requirements for executing those capabilities in stochastic settings. We present an active-learning approach that can effectively interact with a black-box SDM system and learn an interpretable probabilistic model describing its capabilities. Theoretical analysis of the approach identifies the conditions under which the learning process is guaranteed to converge to the correct model of the agent; empirical evaluations on different agents and simulated scenarios show that this approach is few-shot generalizable and can effectively describe the capabilities of arbitrary black-box SDM agents in a sample-efficient manner.

Research paper thumbnail of Discovering User-Interpretable Capabilities of Black-Box Planning Agents

Several approaches have been developed for answering users' specific questions about AI behavior ... more Several approaches have been developed for answering users' specific questions about AI behavior and for assessing their core functionality in terms of primitive executable actions. However, the problem of summarizing an AI agent's broad capabilities for a user is comparatively new. This paper presents an algorithm for discovering from scratch the suite of high-level "capabilities" that an AI system with arbitrary internal planning algorithms/policies can perform. It computes conditions describing the applicability and effects of these capabilities in user-interpretable terms. Starting from a set of user-interpretable state properties, an AI agent, and a simulator that the agent can interact with, our algorithm returns a set of high-level capabilities with their parameterized descriptions. Empirical evaluation on several game-based scenarios shows that this approach efficiently learns descriptions of various types of AI agents in deterministic, fully observable settings. User studies show that such descriptions are easier to understand and reason with than the agent's primitive actions.

Research paper thumbnail of Learning Generalized Models by Interrogating Black-Box Autonomous Agents

arXiv (Cornell University), Dec 29, 2019

This paper develops a new approach for estimating an interpretable, relational model of a black-b... more This paper develops a new approach for estimating an interpretable, relational model of a black-box autonomous agent that can plan and act. Our main contributions are a new paradigm for estimating such models using a rudimentary query interface with the agent and a hierarchical querying algorithm that generates an interrogation policy for estimating the agent's internal model in a user-interpretable vocabulary. Empirical evaluation of our approach shows that despite the intractable search space of possible agent models, our approach allows correct and scalable estimation of interpretable agent models for a wide class of black-box autonomous agents. Our results also show that this approach can use predicate classifiers to learn interpretable models of planning agents that represent states as images.

Research paper thumbnail of Asking the Right Questions: Learning Interpretable Action Models Through Query Answering

arXiv (Cornell University), Dec 29, 2019

This paper develops a new approach for estimating an interpretable, relational model of a black-b... more This paper develops a new approach for estimating an interpretable, relational model of a black-box autonomous agent that can plan and act. Our main contributions are a new paradigm for estimating such models using a rudimentary query interface with the agent and a hierarchical querying algorithm that generates an interrogation policy for estimating the agent's internal model in a user-interpretable vocabulary. Empirical evaluation of our approach shows that despite the intractable search space of possible agent models, our approach allows correct and scalable estimation of interpretable agent models for a wide class of black-box autonomous agents. Our results also show that this approach can use predicate classifiers to learn interpretable models of planning agents that represent states as images.

Research paper thumbnail of Differential Assessment of Black-Box AI Agents

arXiv (Cornell University), Mar 24, 2022

Much of the research on learning symbolic models of AI agents focuses on agents with stationary m... more Much of the research on learning symbolic models of AI agents focuses on agents with stationary models. This assumption fails to hold in settings where the agent's capabilities may change as a result of learning, adaptation, or other post-deployment modifications. Efficient assessment of agents in such settings is critical for learning the true capabilities of an AI system and for ensuring its safe usage. In this work, we propose a novel approach to differentially assess black-box AI agents that have drifted from their previously known models. As a starting point, we consider the fully observable and deterministic setting. We leverage sparse observations of the drifted agent's current behavior and knowledge of its initial model to generate an active querying policy that selectively queries the agent and computes an updated model of its functionality. Empirical evaluation shows that our approach is much more efficient than re-learning the agent model from scratch. We also show that the cost of differential assessment using our method is proportional to the amount of drift in the agent's functionality. * Equal contribution. Alphabetical order.

Research paper thumbnail of Learning Interpretable Models for Black-Box Agents

arXiv (Cornell University), Dec 29, 2019

This paper develops a new approach for learning a STRIPS-like model of a non-stationary black-box... more This paper develops a new approach for learning a STRIPS-like model of a non-stationary black-box autonomous agent that can plan and act. In this approach, the user may ask an autonomous agent a series of questions, which the agent answers truthfully. Our main contribution is an algorithm that generates an interrogation policy in the form of a contingent sequence of questions to be posed to the agent. Answers to these questions are used to learn a minimal, functionally indistinguishable class of agent models. This approach requires a minimal query-answering capability from the agent. Empirical evaluation of our approach shows that despite the intractable space of possible models, our approach can learn interpretable agent models for a class of black-box autonomous agents in a scalable manner.

Research paper thumbnail of Learning Causal Models of Autonomous Agents using Interventions

arXiv (Cornell University), Aug 21, 2021

Research paper thumbnail of Learning User-Interpretable Descriptions of Black-Box AI System Capabilities

arXiv (Cornell University), Jul 28, 2021

Research paper thumbnail of Sample Efficient Paradigms for Personalized Assessment of Taskable AI Systems

Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence

The vast diversity of internal designs of taskable black-box AI systems and their nuanced zones o... more The vast diversity of internal designs of taskable black-box AI systems and their nuanced zones of safe functionality make it difficult for a layperson to use them without unintended side effects. The focus of my dissertation is to develop paradigms that would enable a user to assess and understand the limits of an AI system's safe operability. We develop a personalized AI assessment module that lets an AI system execute instruction sequences in simulators and answer queries about these executions. Our results show that such a primitive query-response capability is sufficient to efficiently derive a user-interpretable model of the system's capabilities in fully observable settings.

Research paper thumbnail of Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing

Research paper thumbnail of JEDAI: A System for Skill-Aligned Explainable Robot Planning

arXiv (Cornell University), Oct 31, 2021

This paper presents JEDAI, an AI system designed for outreach and educational efforts aimed at no... more This paper presents JEDAI, an AI system designed for outreach and educational efforts aimed at non-AI experts. JEDAI features a novel synthesis of research ideas from integrated task and motion planning and explainable AI. JEDAI helps users create high-level, intuitive plans while ensuring that they will be executable by the robot. It also provides users customized explanations about errors and helps improve their understanding of AI planning as well as the limits and capabilities of the underlying robot system.

Research paper thumbnail of Asking the Right Questions: Learning Interpretable Action Models Through Query Answering

Proceedings of the AAAI Conference on Artificial Intelligence

This paper develops a new approach for estimating an interpretable, relational model of a black-b... more This paper develops a new approach for estimating an interpretable, relational model of a black-box autonomous agent that can plan and act. Our main contributions are a new paradigm for estimating such models using a rudimentary query interface with the agent and a hierarchical querying algorithm that generates an interrogation policy for estimating the agent's internal model in a user-interpretable vocabulary. Empirical evaluation of our approach shows that despite the intractable search space of possible agent models, our approach allows correct and scalable estimation of interpretable agent models for a wide class of black-box autonomous agents. Our results also show that this approach can use predicate classifiers to learn interpretable models of planning agents that represent states as images.

Research paper thumbnail of Discovering User-Interpretable Capabilities of Black-Box Planning Agents

Proceedings of the Nineteenth International Conference on Principles of Knowledge Representation and Reasoning

Several approaches have been developed for answering users' specific questions about AI behav... more Several approaches have been developed for answering users' specific questions about AI behavior and for assessing their core functionality in terms of primitive executable actions. However, the problem of summarizing an AI agent's broad capabilities for a user is comparatively new. This paper presents an algorithm for discovering from scratch the suite of high-level "capabilities" that an AI system with arbitrary internal planning algorithms/policies can perform. It computes conditions describing the applicability and effects of these capabilities in user-interpretable terms. Starting from a set of user-interpretable state properties, an AI agent, and a simulator that the agent can interact with, our algorithm returns a set of high-level capabilities with their parameterized descriptions. Empirical evaluation on several game-based scenarios shows that this approach efficiently learns descriptions of various types of AI agents in deterministic, fully observable set...

Research paper thumbnail of Differential Assessment of Black-Box AI Agents

Proceedings of the AAAI Conference on Artificial Intelligence

Much of the research on learning symbolic models of AI agents focuses on agents with stationary m... more Much of the research on learning symbolic models of AI agents focuses on agents with stationary models. This assumption fails to hold in settings where the agent's capabilities may change as a result of learning, adaptation, or other post-deployment modifications. Efficient assessment of agents in such settings is critical for learning the true capabilities of an AI system and for ensuring its safe usage. In this work, we propose a novel approach to differentially assess black-box AI agents that have drifted from their previously known models. As a starting point, we consider the fully observable and deterministic setting. We leverage sparse observations of the drifted agent's current behavior and knowledge of its initial model to generate an active querying policy that selectively queries the agent and computes an updated model of its functionality. Empirical evaluation shows that our approach is much more efficient than re-learning the agent model from scratch. We also sho...

Research paper thumbnail of Can LLMs Converse Formally? Automatically Assessing LLMs in Translating and Interpreting Formal Specifications

arXiv (Cornell University), Mar 27, 2024

Research paper thumbnail of Data Efficient Paradigms for Personalized Assessment of Black-Box Taskable AI Systems

Proceedings of the ... AAAI Conference on Artificial Intelligence, Mar 24, 2024

The vast diversity of internal designs of taskable black-box AI systems and their nuanced zones o... more The vast diversity of internal designs of taskable black-box AI systems and their nuanced zones of safe functionality make it difficult for a layperson to use them without unintended side effects. My dissertation focuses on developing paradigms that enable a user to assess and understand the limits of an AI system's safe operability. We develop a personalized AI assessment module that lets an AI system execute instruction sequences in simulators and answer queries about these executions. Our results show that such a primitive query-response interface is sufficient to efficiently derive a user-interpretable model of a system's capabilities.

Research paper thumbnail of From Reals to Logic and Back: Inventing Symbolic Vocabularies, Actions and Models for Planning from Raw Data

arXiv (Cornell University), Feb 19, 2024

Hand-crafted, logic-based state and action representations have been widely used to overcome the ... more Hand-crafted, logic-based state and action representations have been widely used to overcome the intractable computational complexity of long-horizon robot planning problems, including task and motion planning problems. However, creating such representations requires experts with strong intuitions and detailed knowledge about the robot and the tasks it may need to accomplish in a given setting. Removing this dependency on human intuition is a highly active research area. This paper presents the first approach for autonomously learning generalizable, logic-based relational representations for abstract states and actions starting from unannotated high-dimensional, real-valued robot trajectories. The learned representations constitute auto-invented PDDL-like domain models. Empirical results in deterministic settings show that powerful abstract representations can be learned from just a handful of robot trajectories; the learned relational representations include but go beyond classical, intuitive notions of high-level actions; and that the learned models allow planning algorithms to scale to tasks that were previously beyond the scope of planning without hand-crafted abstractions.

Research paper thumbnail of Epistemic Exploration for Generalizable Planning and Learning in Non-Stationary Settings

arXiv (Cornell University), Feb 12, 2024

This paper introduces a new approach for continual planning and model learning in relational, non... more This paper introduces a new approach for continual planning and model learning in relational, non-stationary stochastic environments. Such capabilities are essential for the deployment of sequential decision-making systems in the uncertain and constantly evolving real world. Working in such practical settings with unknown (and non-stationary) transition systems and changing tasks, the proposed framework models gaps in the agent's current state of knowledge and uses them to conduct focused, investigative explorations. Data collected using these explorations is used for learning generalizable probabilistic models for solving the current task despite continual changes in the environment dynamics. Empirical evaluations on several non-stationary benchmark domains show that this approach significantly outperforms planning and RL baselines in terms of sample complexity. Theoretical results show that the system exhibits desirable convergence properties when stationarity holds.

Research paper thumbnail of Data Efficient Algorithms and Interpretability Requirements for Personalized Assessment of Taskable AI Systems

The vast diversity of internal designs of taskable black-box AI systems and their nuanced zones o... more The vast diversity of internal designs of taskable black-box AI systems and their nuanced zones of safe functionality make it difficult for a layperson to use them without unintended side effects. The focus of my dissertation is to develop algorithms and requirements of interpretability that would enable a user to assess and understand the limits of an AI system's safe operability. We develop an assessment module that lets an AI system execute high-level instruction sequences in simulators and answer the user queries about its execution of sequences of actions. Our results show that such a primitive query-response capability is sufficient to efficiently derive a user-interpretable model of the system in stationary, fully observable, and deterministic settings.

Research paper thumbnail of A comparative study of resource usage for speaker recognition techniques

Resource usage of a software is an important factor to be taken into consideration while developi... more Resource usage of a software is an important factor to be taken into consideration while developing speaker recognition applications for mobile devices. Sometimes usage parameters are considered as important as accuracy of such systems. In this work, we analyze resource utilization in terms of power consumption, memory and space requirements of three standard speaker recognition techniques, viz. GMM-UBM framework, Joint Factor Analysis and i-vectors. Experiments are performed on the MIT MDSVC corpus using the Energy Measurement Library (EML). It is found that though i-vector approach requires more storage space, it is superior to the other two approaches in terms of memory and power consumption, which are critical factors for evaluating software performance in resource constrained mobile devices.

Research paper thumbnail of Autonomous Capability Assessment of Black-Box Sequential Decision-Making Systems

arXiv (Cornell University), Jun 7, 2023

It is essential for users to understand what their AI systems can and can't do in order to use th... more It is essential for users to understand what their AI systems can and can't do in order to use them safely. However, the problem of enabling users to assess AI systems with sequential decision-making (SDM) capabilities is relatively understudied. This paper presents a new approach for modeling the capabilities of black-box AI systems that can plan and act, along with the possible effects and requirements for executing those capabilities in stochastic settings. We present an active-learning approach that can effectively interact with a black-box SDM system and learn an interpretable probabilistic model describing its capabilities. Theoretical analysis of the approach identifies the conditions under which the learning process is guaranteed to converge to the correct model of the agent; empirical evaluations on different agents and simulated scenarios show that this approach is few-shot generalizable and can effectively describe the capabilities of arbitrary black-box SDM agents in a sample-efficient manner.

Research paper thumbnail of Discovering User-Interpretable Capabilities of Black-Box Planning Agents

Several approaches have been developed for answering users' specific questions about AI behavior ... more Several approaches have been developed for answering users' specific questions about AI behavior and for assessing their core functionality in terms of primitive executable actions. However, the problem of summarizing an AI agent's broad capabilities for a user is comparatively new. This paper presents an algorithm for discovering from scratch the suite of high-level "capabilities" that an AI system with arbitrary internal planning algorithms/policies can perform. It computes conditions describing the applicability and effects of these capabilities in user-interpretable terms. Starting from a set of user-interpretable state properties, an AI agent, and a simulator that the agent can interact with, our algorithm returns a set of high-level capabilities with their parameterized descriptions. Empirical evaluation on several game-based scenarios shows that this approach efficiently learns descriptions of various types of AI agents in deterministic, fully observable settings. User studies show that such descriptions are easier to understand and reason with than the agent's primitive actions.

Research paper thumbnail of Learning Generalized Models by Interrogating Black-Box Autonomous Agents

arXiv (Cornell University), Dec 29, 2019

This paper develops a new approach for estimating an interpretable, relational model of a black-b... more This paper develops a new approach for estimating an interpretable, relational model of a black-box autonomous agent that can plan and act. Our main contributions are a new paradigm for estimating such models using a rudimentary query interface with the agent and a hierarchical querying algorithm that generates an interrogation policy for estimating the agent's internal model in a user-interpretable vocabulary. Empirical evaluation of our approach shows that despite the intractable search space of possible agent models, our approach allows correct and scalable estimation of interpretable agent models for a wide class of black-box autonomous agents. Our results also show that this approach can use predicate classifiers to learn interpretable models of planning agents that represent states as images.

Research paper thumbnail of Asking the Right Questions: Learning Interpretable Action Models Through Query Answering

arXiv (Cornell University), Dec 29, 2019

This paper develops a new approach for estimating an interpretable, relational model of a black-b... more This paper develops a new approach for estimating an interpretable, relational model of a black-box autonomous agent that can plan and act. Our main contributions are a new paradigm for estimating such models using a rudimentary query interface with the agent and a hierarchical querying algorithm that generates an interrogation policy for estimating the agent's internal model in a user-interpretable vocabulary. Empirical evaluation of our approach shows that despite the intractable search space of possible agent models, our approach allows correct and scalable estimation of interpretable agent models for a wide class of black-box autonomous agents. Our results also show that this approach can use predicate classifiers to learn interpretable models of planning agents that represent states as images.

Research paper thumbnail of Differential Assessment of Black-Box AI Agents

arXiv (Cornell University), Mar 24, 2022

Much of the research on learning symbolic models of AI agents focuses on agents with stationary m... more Much of the research on learning symbolic models of AI agents focuses on agents with stationary models. This assumption fails to hold in settings where the agent's capabilities may change as a result of learning, adaptation, or other post-deployment modifications. Efficient assessment of agents in such settings is critical for learning the true capabilities of an AI system and for ensuring its safe usage. In this work, we propose a novel approach to differentially assess black-box AI agents that have drifted from their previously known models. As a starting point, we consider the fully observable and deterministic setting. We leverage sparse observations of the drifted agent's current behavior and knowledge of its initial model to generate an active querying policy that selectively queries the agent and computes an updated model of its functionality. Empirical evaluation shows that our approach is much more efficient than re-learning the agent model from scratch. We also show that the cost of differential assessment using our method is proportional to the amount of drift in the agent's functionality. * Equal contribution. Alphabetical order.

Research paper thumbnail of Learning Interpretable Models for Black-Box Agents

arXiv (Cornell University), Dec 29, 2019

This paper develops a new approach for learning a STRIPS-like model of a non-stationary black-box... more This paper develops a new approach for learning a STRIPS-like model of a non-stationary black-box autonomous agent that can plan and act. In this approach, the user may ask an autonomous agent a series of questions, which the agent answers truthfully. Our main contribution is an algorithm that generates an interrogation policy in the form of a contingent sequence of questions to be posed to the agent. Answers to these questions are used to learn a minimal, functionally indistinguishable class of agent models. This approach requires a minimal query-answering capability from the agent. Empirical evaluation of our approach shows that despite the intractable space of possible models, our approach can learn interpretable agent models for a class of black-box autonomous agents in a scalable manner.

Research paper thumbnail of Learning Causal Models of Autonomous Agents using Interventions

arXiv (Cornell University), Aug 21, 2021

Research paper thumbnail of Learning User-Interpretable Descriptions of Black-Box AI System Capabilities

arXiv (Cornell University), Jul 28, 2021

Research paper thumbnail of Sample Efficient Paradigms for Personalized Assessment of Taskable AI Systems

Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence

The vast diversity of internal designs of taskable black-box AI systems and their nuanced zones o... more The vast diversity of internal designs of taskable black-box AI systems and their nuanced zones of safe functionality make it difficult for a layperson to use them without unintended side effects. The focus of my dissertation is to develop paradigms that would enable a user to assess and understand the limits of an AI system's safe operability. We develop a personalized AI assessment module that lets an AI system execute instruction sequences in simulators and answer queries about these executions. Our results show that such a primitive query-response capability is sufficient to efficiently derive a user-interpretable model of the system's capabilities in fully observable settings.

Research paper thumbnail of Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing

Research paper thumbnail of JEDAI: A System for Skill-Aligned Explainable Robot Planning

arXiv (Cornell University), Oct 31, 2021

This paper presents JEDAI, an AI system designed for outreach and educational efforts aimed at no... more This paper presents JEDAI, an AI system designed for outreach and educational efforts aimed at non-AI experts. JEDAI features a novel synthesis of research ideas from integrated task and motion planning and explainable AI. JEDAI helps users create high-level, intuitive plans while ensuring that they will be executable by the robot. It also provides users customized explanations about errors and helps improve their understanding of AI planning as well as the limits and capabilities of the underlying robot system.

Research paper thumbnail of Asking the Right Questions: Learning Interpretable Action Models Through Query Answering

Proceedings of the AAAI Conference on Artificial Intelligence

This paper develops a new approach for estimating an interpretable, relational model of a black-b... more This paper develops a new approach for estimating an interpretable, relational model of a black-box autonomous agent that can plan and act. Our main contributions are a new paradigm for estimating such models using a rudimentary query interface with the agent and a hierarchical querying algorithm that generates an interrogation policy for estimating the agent's internal model in a user-interpretable vocabulary. Empirical evaluation of our approach shows that despite the intractable search space of possible agent models, our approach allows correct and scalable estimation of interpretable agent models for a wide class of black-box autonomous agents. Our results also show that this approach can use predicate classifiers to learn interpretable models of planning agents that represent states as images.

Research paper thumbnail of Discovering User-Interpretable Capabilities of Black-Box Planning Agents

Proceedings of the Nineteenth International Conference on Principles of Knowledge Representation and Reasoning

Several approaches have been developed for answering users' specific questions about AI behav... more Several approaches have been developed for answering users' specific questions about AI behavior and for assessing their core functionality in terms of primitive executable actions. However, the problem of summarizing an AI agent's broad capabilities for a user is comparatively new. This paper presents an algorithm for discovering from scratch the suite of high-level "capabilities" that an AI system with arbitrary internal planning algorithms/policies can perform. It computes conditions describing the applicability and effects of these capabilities in user-interpretable terms. Starting from a set of user-interpretable state properties, an AI agent, and a simulator that the agent can interact with, our algorithm returns a set of high-level capabilities with their parameterized descriptions. Empirical evaluation on several game-based scenarios shows that this approach efficiently learns descriptions of various types of AI agents in deterministic, fully observable set...

Research paper thumbnail of Differential Assessment of Black-Box AI Agents

Proceedings of the AAAI Conference on Artificial Intelligence

Much of the research on learning symbolic models of AI agents focuses on agents with stationary m... more Much of the research on learning symbolic models of AI agents focuses on agents with stationary models. This assumption fails to hold in settings where the agent's capabilities may change as a result of learning, adaptation, or other post-deployment modifications. Efficient assessment of agents in such settings is critical for learning the true capabilities of an AI system and for ensuring its safe usage. In this work, we propose a novel approach to differentially assess black-box AI agents that have drifted from their previously known models. As a starting point, we consider the fully observable and deterministic setting. We leverage sparse observations of the drifted agent's current behavior and knowledge of its initial model to generate an active querying policy that selectively queries the agent and computes an updated model of its functionality. Empirical evaluation shows that our approach is much more efficient than re-learning the agent model from scratch. We also sho...