Adaptive Symbolic Control for Finite-State Transition Systems With Grammatical Inference (original) (raw)

Abstract

This paper presents an approach that brings together game theory with grammatical inference and discrete abstractions in order to synthesize control strategies for hybrid dynamical systems performing tasks in partially unknown but rule-governed adversarial environments. The combined formulation guarantees that a system specification is met if (a) the true model of the environment is in the class of models inferable from a positive presentation, (b) a characteristic sample is observed, and (c) the task specification is satisfiable given the capabilities of the system (agent) and the environment.

Figures (10)

Fig. 1: The architecture of hybrid agentic planning and control with a module for grammatical inference.

Fig. 1: The architecture of hybrid agentic planning and control with a module for grammatical inference.

the exact language presented to the learner, irrespectively if this language can also be generated

the exact language presented to the learner, irrespectively if this language can also be generated

pair of closed doors. Fig. 3: The non-cooperative game used in this case study. Figure 3(a) is a graphical depiction of the triangular apartment game, while Fig. 3(b) shows a physical realization of the scenario, with a Knepera II miniature mobile robot in the role of the hybrid agent. The robot localizes itself and observes which doors are closed (door closure implemented manually using the yellow caution cones) through a VICON™ motion capture system. The grammatical inference module and the strategy computation algorithm have been implemented in python, which communicates with the control for the robot (through Matlab™) over a serial link. Fig. 3: The non-cooperative game used in this case study. Figure 3(a) is a graphical depiction of the triangular TABLE I: Some possible rules for the adversary (controlling the doors): at each round, the environment either grammatical inference module and the strategy computation algorithm have been implemented in python, which

pair of closed doors. Fig. 3: The non-cooperative game used in this case study. Figure 3(a) is a graphical depiction of the triangular apartment game, while Fig. 3(b) shows a physical realization of the scenario, with a Knepera II miniature mobile robot in the role of the hybrid agent. The robot localizes itself and observes which doors are closed (door closure implemented manually using the yellow caution cones) through a VICON™ motion capture system. The grammatical inference module and the strategy computation algorithm have been implemented in python, which communicates with the control for the robot (through Matlab™) over a serial link. Fig. 3: The non-cooperative game used in this case study. Figure 3(a) is a graphical depiction of the triangular TABLE I: Some possible rules for the adversary (controlling the doors): at each round, the environment either grammatical inference module and the strategy computation algorithm have been implemented in python, which

Fig. 4: Semiautomata for the agent (left) and for a fragment of the environment (right). In Aj, the states are the rooms and the transitions are labeled with the rooms that the agent is to enter. For Ao, the states represent the pairs of doors that are currently closed and a transition xy indicates the pair of doors x,y are to be closed. By assigning J; = Q, and Jy = Qo, the game can start with any state in Q; x Q2 x {1}

Fig. 4: Semiautomata for the agent (left) and for a fragment of the environment (right). In Aj, the states are the rooms and the transitions are labeled with the rooms that the agent is to enter. For Ao, the states represent the pairs of doors that are currently closed and a transition xy indicates the pair of doors x,y are to be closed. By assigning J; = Q, and Jy = Qo, the game can start with any state in Q; x Q2 x {1}

if A, o Ay, while a fragment of the game automaton G is shown in Fig. 7. ‘ig. 6: Fragment of turn-based product P = A; 0 Az = (Qy, U1 U Xo, T,). State (r, d;d2,c) means the agent is in room r, doors {d,, d2} are closed and the Boolean variable keeping track of whose turn it is set to c.

if A, o Ay, while a fragment of the game automaton G is shown in Fig. 7. ‘ig. 6: Fragment of turn-based product P = A; 0 Az = (Qy, U1 U Xo, T,). State (r, d;d2,c) means the agent is in room r, doors {d,, d2} are closed and the Boolean variable keeping track of whose turn it is set to c.

Fig. 7: Fragment of the game automaton G = (Q,%) U Y2,T7,Qo,F) for the doorrobot game, where Qo = {(u,@, 1, qs) | ga € Th, q2 € 12, qs HSN € {12394} and Fo {(q1, 92,0, 1234) | (q1; G2, 0) € Qp}, note that upon initialization of a game, the state of A, (the room occupied by the robot) determines the choice of initial state in A (tho mnm vwvicitan hy tho mbhnt ) in A, (the room visited by the robot.)

Fig. 7: Fragment of the game automaton G = (Q,%) U Y2,T7,Qo,F) for the doorrobot game, where Qo = {(u,@, 1, qs) | ga € Th, q2 € 12, qs HSN € {12394} and Fo {(q1, 92,0, 1234) | (q1; G2, 0) € Qp}, note that upon initialization of a game, the state of A, (the room occupied by the robot) determines the choice of initial state in A (tho mnm vwvicitan hy tho mbhnt ) in A, (the room visited by the robot.)

Num of games Num of wins

Num of games Num of wins

Loading...

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

References (49)

  1. H. Tanner, J. Fu, C. Rawal, J. Piovesan, and C. Abdallah, "Finite abstractions for hybrid systems with stable continuous dynamics," Discrete Event Dynamic Systems, vol. 22, pp. 83-99, 2012.
  2. G. E. Fainekos, A. Girard, H. Kress-Gazit, and G. J. Pappas, "Temporal logic motion planning for dynamic robots," Automatica, vol. 45, no. 2, pp. 343-352, Feb. 2009.
  3. C. de la Higuera, Grammatical Inference: Learning Automata and Grammars. Cambridge University Press, 2010.
  4. W. Zielonka, "Infinite games on finitely coloured graphs with applications to automata on infinite trees," Theoretical Computer Science, vol. 200, no. 1-2, pp. 135-183, 1998.
  5. P. J. Ramadge and W. M. Wonham, "Supervisory Control of a Class of Discrete Event Processes," SIAM Journal on Control and Optimization, vol. 25, no. 1, pp. 206-230, 1987.
  6. J. Knight and B. Luense, "Control theory, modal logic, and games," in Hybrid Systems: Computation and Control, ser. Lecture Notes in Computer Science, P. Antsaklis, W. Kohn, A. Nerode, and S. Sastry, Eds. Springer Berlin / Heidelberg, 1997, vol. 1273, pp. 160-173.
  7. E. Grädel, W. Thomas, and T. Wilke, Eds., Automata logics, and infinite games: a guide to current research. New York, NY, USA: Springer-Verlag New York, Inc., 2002.
  8. N. Piterman and A. Pnueli, "Synthesis of reactive(1) designs," in In Proceedings of Verification, Model Checking, and Abstract Interpretation. Springer, 2006, pp. 364-380.
  9. C. Belta, A. Bicchi, M. Egerstedt, E. Frazzoli, E. Klavins, and G. Pappas, "Symbolic planning and control of robot motion," IEEE Robotics Automation Magazine, vol. 14, no. 1, pp. 61-70, 2007.
  10. M. Lahijanian, J. Wasniewski, S. Andersson, and C. Belta, "Motion planning and control from temporal logic specifications with probabilistic satisfaction guarantees," in IEEE International Conference on Robotics and Automation, 2010, pp. 3227- 3232.
  11. A. LaViers, M. Egerstedt, Y. Chen, and C. Belta, "Automatic generation of balletic motions," in Proceedings of the 2011 IEEE/ACM Second International Conference on Cyber-Physical Systems, Washington, DC, USA, 2011, pp. 13-21.
  12. T. Wongpiromsarn, U. Topcu, and R. M. Murray, "Receding horizon control for temporal logic specifications," in Hybrid Systems: Computation and Control, K. H. Johansson and W. Yi, Eds. New York, NY, USA: ACM, 2010, pp. 101-110.
  13. C. J. Tomlin, I. Mitchell, A. M. Bayen, and M. Oishi, "Computational techniques for the verification of hybrid systems," Proceedings of the IEEE, vol. 91, no. 7, pp. 986-1001, july 2003.
  14. M. Kloetzer and C. Belta, "Dealing with nondeterminism in symbolic control," in Hybrid Systems: Computation and Control, ser. Lecture Notes in Computer Science, M. Egerstedt and B. Mishra, Eds. Springer, 2008, pp. 287-300.
  15. H. Kress-Gazit, G. E. Fainekos, and G. J. Pappas, "Temporal-logic-based reactive mission and motion planning," IEEE Transactions on Robotics, vol. 25, no. 6, pp. 1370-1381, dec. 2009.
  16. H. Kress-Gazit, T. Wongpiromsarn, and U. Topcu, "Correct, reactive, high-level robot control," Robotics Automation Magazine, IEEE, vol. 18, no. 3, pp. 65 -74, sept. 2011.
  17. K. J. Astrom and B. Wittenmark, Adaptive Control. Addison-Wesley, 1995.
  18. S. Sastry and M. Bodson, Adaptive Control. Prentice Hall, 1989.
  19. M. J. Matarić, "Reinforcement learning in the multi-robot domain," Autonomous Robots, vol. 4, no. 1, pp. 73-83, 1997.
  20. J. Peters, S. Vijayakumar, and S. Schaal, "Reinforcement learning for humanoid robotics," in Proceedings of the IEEE-RAS International Conference on Humanoid Robots, 2003.
  21. E. Brunskill, B. R. Leffler, L. Li, M. L. Littman, and N. Roy, "Provably efficient learning with typed parametric models," Journal of Machine Learning Research, vol. 10, pp. 1955-1988, 2009.
  22. P. Abbeel, A. Coates, and A. Y. Ng, "Autonomous helicopter aerobatics through apprenticeship learning," International Journal of Robotics Research, vol. 29, no. 13, pp. 1608-1639, 2010.
  23. A. Hamdi-Cherif and C. Kara-Mohammed, "Grammatical inference methodology for control systems," WSEAS Transaction on Computers, vol. 8, no. 4, pp. 610-619, Apr. 2009.
  24. C. B. Yushan Chen, Jana Tumova, "LTL robot motion control based on automata learning of environmental dynamics," in IEEE International Conference on Robotics and Automation, Saint Paul, MN, USA, 2012.
  25. E. M. Gold, "Language identification in the limit," Information and Control, vol. 10, no. 5, pp. 447-474, 1967.
  26. J. Heinz, "String extension learning," in Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, Uppsala, Sweden, July 2010, pp. 897-906.
  27. A. Kasprzik and T. Kötzing, "String extension learning using lattices," in Language and Automata Theory and Applications: 4th International Conference, LATA 2010, ser. Lecture Notes in Computer Science, C. Martin-Vide, H. Fernau, and A. H. Dediu, Eds., vol. 6031. Trier, Germany: Springer, 2010, pp. 380-391.
  28. R. McNaughton and S. Papert, Counter-Free Automata. MIT Press, 1971.
  29. A. De Luca and A. Restivo, "A characterization of strictly locally testable languages and its application to subsemigroups of a free semigroup," Information and Control, vol. 44, no. 3, pp. 300-319, Mar. 1980.
  30. P. Garcia, E. Vidal, and J. Oncina, "Learning locally testable languages in the strict sense," in Proceedings of the Workshop on Algorithmic Learning Theory, 1990, pp. 325-338.
  31. J. Rogers, "Cognitive complexity in the sub-regular realm," UCLA Colloquium, Oct. 2010.
  32. J. Lygeros, K. Johansson, S. Simić, and S. Sastry, "Dynamical properties of hybrid automata," IEEE Transactions on Automatic Control, vol. 48, no. 1, pp. 2-17, 2003.
  33. H. Enderton, A Mathematical Introduction to Logic. Academic Press, 1972.
  34. H. Khalil, Nonlinear Systems, 3rd ed. Prentice Hall, 2002.
  35. C. Stirling, "Modal and temporal logics for processes," in Logics for concurency: structure vs automata, F. Moller and G. Birtwistle, Eds. Springer, 1996.
  36. E. M. Clarke Jr., O. Grumberg, and D. A. Peled, Model checking. MIT Press, 1999.
  37. D. Perrin and J. Éric Pin, Infinite words:automata, semigroups, logic and games. Elsevier, 2004.
  38. W. Thomas, "Infinite games and verification (extended abstract of a tutorial)," in Proceedings of the 14th International Conference on Computer Aided Verification, ser. CAV '02. London, UK, UK: Springer-Verlag, 2002, pp. 58-64.
  39. U. Frith and C. Frith, "Development and neurophysiology of mentalizing," Philosophical Transactions of the Royal Society B Biological Sciences, no. 358, pp. 459-473, 2003.
  40. D. Premack and G. Woodruff, "Does the chimpanzee have a theory of mind?" Behavioral and Brain Sciences, vol. 1, no. 04, pp. 515-526, 1978.
  41. P. Caron, "LANGAGE: a maple package for automaton characterization of regular languages," in Automata Implementation, ser. Lecture Notes in Computer Science, D. Wood and S. Yu, Eds. Springer, 1998, vol. 1436, pp. 46-55.
  42. J. Heinz, "Inductive learning of phonotactic patterns," Ph.D. dissertation, University of California, Los Angeles, 2007.
  43. J. Fu and H. G. Tanner, "Optimal planning on register automata," in American Control Conference, Jun 2012 (to appear).
  44. A. K. Das, R. Fierro, V. Kumar, B. Southall, J. Spletzer, and C. J. Taylor, "Real-time vision-based control of a nonholonomic mobile robot," in Proceedings of the IEEE International Conference on Robotics and Automation, 2001, pp. 1714-1719.
  45. D. C. Conner, H. Choset, and A. A. Rizzi, "Flow-through policies for hybrid controller synthesis applied to fully actuated systems," IEEE Transactions on Robotics, vol. 25, no. 1, pp. 136-146, 2009.
  46. H. Kress-Gazit, G. E. Fainekos, and G. J. Pappas, "Where's Waldo? sensor-based temporal logic motion planning," in Proceedings of the IEEE International Conference on Robotics and Automation, 2007, pp. 3116-3121.
  47. D. C. Conner, "Integrating planning and control for constrained dynamical systems," Ph.D. dissertation, Carnegie Mellon University, December 2007.
  48. R. Mazala, "Infinite games," in Automata, Logics, and Infinite Games, 2001, pp. 23-42.
  49. K. Chatterjee, T. Henzinger, and N. Piterman, "Algorithms for Büchi games," in Games in Design and Verification, 2006.