Yundong Cai - Academia.edu (original) (raw)
Papers by Yundong Cai
Interactive storytelling, emerging with virtual reality technol-ogy, has attracted a lot of resea... more Interactive storytelling, emerging with virtual reality technol-ogy, has attracted a lot of research interests in recent years. In order to bridge the gap between story generation and story presentation, some hybrid narrative models are raised. In this paper, we propose a new hybrid interactive storytelling archi-tecture S-MADE. It combines the story plot generation and character performance through goal execution and decompo-sition mechanism. Fuzzy Cognitive Goal Net (FCGN) is a goal modeling tool for modeling the story scenario and char-acter behaviors. The goal decomposition algorithm enables for director agent to dispatch acts to characters easily. With this system, dynamic storylines as well as character behav-iors are created in real time simultaneously, which is hard to be achieved in traditional approaches. Audiences are able to experience different perspectives through interactions with director and characters.
Authoring Goal Oriented Educational Game in Virtual Learning Environment
This paper uses learning goal as a continuous thread to organize the educational game authoring p... more This paper uses learning goal as a continuous thread to organize the educational game authoring process in order to smoothly integrate learning contents into the game designing process. The proposed authoring system has been used in a practical virtual learning environment Virtual Singapora. This goal oriented system can not only facilitate game designers during the authoring process, but also provide students with personalized control of their own
I would like to express my sincere appreciation to the following people for their support, encour... more I would like to express my sincere appreciation to the following people for their support, encouragement and help during my Ph.D study at Nanyang Technological University. First and foremost, I would like to take this opportunity to pay a high tribute to my PhD supervisor Dr. Miao Chun Yan, for her continuous support and numerous guidance at my research work as well as life in the academic years. Dr. Miao provides great freedom for my research, and was always willing to listen and share her experiences with me. Moreover, she often discussed with me about potential research directions over time, which inspired me in the area of agent-mediated interactive storytelling. I'd also like to thank my co-supervisor, Dr. Tan Ah-Hwee, who is most helpful by providing insightful comments and reviewing my papers. Moreover, special thanks go to Dr. Shen Zhiqi, for a lot of inspirations from him in the discussions and technical guidance. Thanks also go to Emerging Research Lab, School of Computer Engineering, Nanyang Technological University, for the financial support and computing facilities, and helpful lab technicians Wong Kia Yan and Thin Nandar Soe. Dr. Weng Jianshu gave me strong support, great advice and discreet work to improve my thesis. I also would like to thank other team members at developing the projects of serious games and interactive storytelling, who are Benny Tan,
Autonomous Agents & Multiagent Systems/Agent Theories, Architectures, and Languages, 2009
In this paper, we introduce an novel emotional agent system in 3D virtual world based on OCC (Ort... more In this paper, we introduce an novel emotional agent system in 3D virtual world based on OCC (Ortony, Clore and Collins) theory, FCM (Fuzzy Cognitive Map) and GoalNet. The agent system is designed based on Goal Net model. Emotional modeling and decision making are based on OCC and FCM inference. Emotions modeled by the OCC model are incorporated into FCM
2007 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Workshops, 2007
Personalization in virtual learning environments is the system ability to provide individualizati... more Personalization in virtual learning environments is the system ability to provide individualization and a set of personalized services such as personalized content management, learner model, or adaptive instant interaction. The intelligent agent technology has potential regarding the creation of such personalized, adaptive and interactive elearning applications. However, most of the available solutions have so far focused on porting existing courses with traditional teaching methods onto the virtual environments, making them available in an attractive animated interface without any fine-tuning and adaptation to the learner needs. This paper proposes a novel market-inspired collaboration model where the agents are self-interested autonomic elements collaborate to achieve a comprehensive learner model. Mentor agent makes decisions on top of a Dempster-Shafer belief accumulation to help student whenever she believes student has lost the clues and needs help. Proposed architecture is validated by applying on a sample agent augmented virtual environment designed to engage and motivate students at the lower secondary level in Singapore. Extensive experiments illustrate the effectiveness of the proposed interaction model where students have found the mentor agent as believable as a virtual teacher.
Proceedings of the 15th international conference on Intelligent user interfaces - IUI '10, 2010
Riding on the back of the rapid expansion of the Internet, online virtual worlds which combine th... more Riding on the back of the rapid expansion of the Internet, online virtual worlds which combine the prowess of interactive digital media and social networks have attained a high level of acceptance among the Net Generation users. This development prompted researchers to look into the potential of embedding learning contents into virtual worlds to create virtual learning environments (VLEs) that suit the need of the Net Generation learners. However, the special characteristics of virtual worlds that make them popular also pose great challenges to educators who wish to leverage their power. These challenges call for more sophisticated human computer interaction (HCI) mechanisms to assist learners to navigate the intriguing landscape of VLEs. In this paper, we demonstrate a teachable remembrance agent which acts as an intelligent user interface to provide innovative ways for students to interact with VLEs.
Lecture Notes in Computer Science, 2011
Interactive virtual storytelling attracts a lot of research interests to create engaging, interac... more Interactive virtual storytelling attracts a lot of research interests to create engaging, interactive and dynamic storytelling. However, currently there is a big gap between story plot modeling to believable character modeling and it is still a great challenge to create an interactive storytelling system from text-based stories for novice. In this paper, we propose an Agent-Oriented Methodology for Interactive Storytelling (AOMIS) to guide novice to generate interactive storytelling system. In this methodology, interactive storytelling system is constructed as a multi-agent system of director agent and character agents. The interactive storytelling is simulated as the goal pursuing of the agents affecting by user interactions and context changes. By viewing the interactive storytelling in the perspective of goal-oriented agents, our methodology is a hybrid of plot-oriented interactive storytelling and character-based interactive storytelling.
Lecture Notes in Computer Science, 2012
Virtual laboratory removes the barrier to conduct the experiments which are difficult to achieve ... more Virtual laboratory removes the barrier to conduct the experiments which are difficult to achieve in the real world. A lot of researches have been done on empowering the functionalities of virtual laboratories. However, it is still a challenge to provide a personalized learning experience in the virtual learning environment for the students. In our paper, we propose an agent-based virtual learning environment that helps the students to conduct the virtual experiments through the simulations and immersive role-playing. Agents are able to provide better instructions and stimulate the students conduct the deeper learning. Moreover, they monitor the learning process of the students, which is helpful to provide the customized learning in real-time. We use the methodology to teach the plant transportation for secondary school students in school and receives positive comments.
Virtual Actors in a Collaborative Story Creation System/9 António Brisson, Ana Paiva S-MADE: Interactive Storytelling Architecture through Goal Execution and Decomposition/17
Page 1. Contents Preface / vii Brian S. Magerko and Mark O. Riedl Rational Dialog in Interactive ... more Page 1. Contents Preface / vii Brian S. Magerko and Mark O. Riedl Rational Dialog in Interactive Games / 1 Maria Arinbjarnar Are We Telling the Same Story? Balancing Real and Virtual Actors in a Collaborative Story Creation System / 9 Ant��nio Brisson, Ana Paiva S-MADE: Interactive Storytelling Architecture through Goal Execution and Decomposition / 17 Yundong Cai, Zhiqi Shen, Chunyan Miao, Ah-Hwee Tan The Story Molecule: Narrative as Information / 21 Beth Cardier How Can We Be Serious in a Game? ...
Agent Mediated Interactive Storytelling in Virtual Environment
i-CAVE: Intelligent Agent-Augmented CAVE System
Cave Automatic Virtual Environment (CAVE) is a virtual reality environment with immersive 3D expe... more Cave Automatic Virtual Environment (CAVE) is a virtual reality environment with immersive 3D experience and engaging interactions, which would be a very good medium of entertainment, training or exercise for elderly. However, the raw CAVE system doesn’t provide semantic understanding of players’ behaviors and emotions in real-time, which leads to a challenge for application developers to create a personalized virtual environment for various players. In this paper we proposed an intelligent agentaugmented CAVE system, namely i-CAVE, to bridge the gap between application developers and end players, especially the elderly which has the special requirements of interactions. We have implemented the system with EON Studio in a Live CAVE environment and a case study called “Walking with Dinosaurs”. The interviewed players give very positive feedback on the immersion and engaging interactions. KeywordsCAVE, semantic information, agent
Cognitive Teachable Agent in Virtual Learning Environment with Fuzzy Cognitive Map
I would like to express my sincere appreciation to the following people for their support, encour... more I would like to express my sincere appreciation to the following people for their support, encouragement and help during my Ph.D study at Nanyang Technological University. First and foremost, I would like to take this opportunity to pay a high tribute to my PhD supervisor Dr. Miao Chun Yan, for her continuous support and numerous guidance at my research work as well as life in the academic years. Dr. Miao provides great freedom for my research, and was always willing to listen and share her experiences with me. Moreover, she often discussed with me about potential research directions over time, which inspired me in the area of agent-mediated interactive storytelling. I'd also like to thank my co-supervisor, Dr. Tan Ah-Hwee, who is most helpful by providing insightful comments and reviewing my papers. Moreover, special thanks go to Dr. Shen Zhiqi, for a lot of inspirations from him in the discussions and technical guidance. Thanks also go to Emerging Research Lab, School of Computer Engineering, Nanyang Technological University, for the financial support and computing facilities, and helpful lab technicians Wong Kia Yan and Thin Nandar Soe. Dr. Weng Jianshu gave me strong support, great advice and discreet work to improve my thesis. I also would like to thank other team members at developing the projects of serious games and interactive storytelling, who are Benny Tan,
Virtual laboratory removes the barrier to conduct the experiments which are difficult to achieve ... more Virtual laboratory removes the barrier to conduct the experiments which are difficult to achieve in the real world. A lot of researches have been done on empowering the functionalities of virtual laboratories. However, it is still a challenge to provide a personalized learning experience in the virtual learning environment for the students. In our paper, we propose an agent-based virtual learning environment that helps the students to conduct the virtual experiments through the simulations and immersive role-playing. Agents are able to provide better instructions and stimulate the students conduct the deeper learning. Moreover, they monitor the learning process of the students, which is helpful to provide the customized learning in real-time. We use the methodology to teach the plant transportation for secondary school students in school and receives positive comments.
Existing Parkinson's Disease (PD) diagnosis relies heavily on doctors' observations combined with... more Existing Parkinson's Disease (PD) diagnosis relies heavily on doctors' observations combined with neurological exams. Such a technique is often inconvenient, infrequent, and subjective, which leads to a high misdiagnosis rate. As several cardinal symptoms of PD require long term observations, a technology platform which allows potential PD patients to exhibit related behaviors in a natural setting over long period of time is needed. In this paper, we describe an agentbased game for the predictive diagnosis of PD. Agents in this tablet based game provide companionship, encouragement, and analysis capabilities to help retain users' interest, and analyze their risk of developing PD symptoms based on their in-game behavior. The game has been launched together with a world renowned PD research centre for clinical trial. It can potentially provide a new dimension of longitudinal interactive behavior data to assist early and more accurate PD diagnosis in the future.
Internet of Things Based Data Driven Storytelling for Supporting Social Connections
2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing, 2013
ABSTRACT The empty nest syndrome has become a significant modern social problem. Recent research ... more ABSTRACT The empty nest syndrome has become a significant modern social problem. Recent research has enhanced physical health monitoring for the elderly in smart home environment. However, the social needs of the empty nesters have been less supported in smart home designs. A compelling story about the elderly can make the adult children emotionally attached to their elderly parents and establish a healthy social connection between them. In this work, we propose a data-driven story generation system based on the Internet of Things (IoT). IoT based techniques enable the real time collection of the elderly's environment data through various sensors located in the home. The story generation system can weave the information mined from the sensor data into a story with high persuasive and emotion arousing power. Our system automatically collects raw sensor data on the cloud server and maps them to values of semantically meaningful context variables through activity functions. A director agent organizes the story plot with a Goal Net based approach and dynamically generates storylines based on a set of scene selection rules. The proposed system is carefully studied with a simulated smart home environment.
Technologies for E-Learning and Digital Entertainment
Interactive storytelling in the virtual environment attracts a lot of research interests in recen... more Interactive storytelling in the virtual environment attracts a lot of research interests in recent years. Story plot and character are two most important elements of a story. Based on these two elements, currently there are two research directions: plot-based and character-based interactive storytelling. However, plot-based approach lacks the refinement of character behaviors as character-based approach. On the other side, character-based approach does not follow a well organized story plot so that the moral of the story might be distorted. Therefore, there is a need to develop an integrated framework to achieve the balance between conveying story moral and enhancing the modeling of character behaviors. In this paper, we propose a hybrid system of the plot-based and character-based approaches. It is constructed as a multi-agent system (MAS), which involves a scriptwriter agent, a director agent, virtual actor agents and other support agents. Fuzzy Cognitive Goal Net (FCGN) is used by the scriptwriter agent to generate story plot, which includes various meaningful storylines. The director agent selects the storyline dynamically and dispatches the scenes to the virtual character agents as behaviors through the decomposing algorithms. With the system, dynamic storylines and character behaviors are generated in realtime based on the audience interactions and context changes. The audiences are able to experience different levels of interactions as an observer, a character or a director. A case study prototype called "Mystery Illness Investigation at Nanyang Town" has been implemented based on the proposed system.
2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence), 2008
To generate a believable and dynamic virtual world is a great challenge in interactive storytelli... more To generate a believable and dynamic virtual world is a great challenge in interactive storytelling. In this paper, we propose a model, namely Evolutionary Fuzzy Cognitive Map (E-FCM), to model the dynamic causal relationships among different context variables. As an extension to conventional FCM, E-FCM models not only the fuzzy causal relationships among the variables, but also the probabilistic property of causal relationships, and asynchronous activity update of the concepts. With this model, the context variables evolve in a dynamic and uncertain manner with the according evolving time. As a result, the virtual world is presented more realistically and dynamically.
Lecture Notes in Computer Science, 2012
As treasure houses of human history and knowledge with rich repositories on cultural heritage, mu... more As treasure houses of human history and knowledge with rich repositories on cultural heritage, museums and libraries have advantages of presence experience and powerful searching abilities respectively. We propose in this paper an integrated design of virtual museum and digital library in the preservation of cultural heritage of the Chinese "qipao", as an example of preservation of cultural heritage resources in digital space.
Crowdsourcing Metadata Schema Generation for Chinese-Style Costume Digital Library
Lecture Notes in Computer Science, 2012
Digital library (DL) has been used as a very important tool for cultural preservation and appreci... more Digital library (DL) has been used as a very important tool for cultural preservation and appreciation. However, it is a great challenge to generate effective and efficient metadata schemas to describe digital artefacts once common metadata schema standards are not enough. In this paper, we propose a crowdsourcing metadata schema generation methodology and use it to create a new metadata schema, "Costume Core" (CC) for Chinese-style costume-related heritage resources, which could be used by other costume digital libraries. We then conducted an initial survey to get users’ feedback on the ranked importance of the metadata elements in the CC schema.
Interactive storytelling, emerging with virtual reality technol-ogy, has attracted a lot of resea... more Interactive storytelling, emerging with virtual reality technol-ogy, has attracted a lot of research interests in recent years. In order to bridge the gap between story generation and story presentation, some hybrid narrative models are raised. In this paper, we propose a new hybrid interactive storytelling archi-tecture S-MADE. It combines the story plot generation and character performance through goal execution and decompo-sition mechanism. Fuzzy Cognitive Goal Net (FCGN) is a goal modeling tool for modeling the story scenario and char-acter behaviors. The goal decomposition algorithm enables for director agent to dispatch acts to characters easily. With this system, dynamic storylines as well as character behav-iors are created in real time simultaneously, which is hard to be achieved in traditional approaches. Audiences are able to experience different perspectives through interactions with director and characters.
Authoring Goal Oriented Educational Game in Virtual Learning Environment
This paper uses learning goal as a continuous thread to organize the educational game authoring p... more This paper uses learning goal as a continuous thread to organize the educational game authoring process in order to smoothly integrate learning contents into the game designing process. The proposed authoring system has been used in a practical virtual learning environment Virtual Singapora. This goal oriented system can not only facilitate game designers during the authoring process, but also provide students with personalized control of their own
I would like to express my sincere appreciation to the following people for their support, encour... more I would like to express my sincere appreciation to the following people for their support, encouragement and help during my Ph.D study at Nanyang Technological University. First and foremost, I would like to take this opportunity to pay a high tribute to my PhD supervisor Dr. Miao Chun Yan, for her continuous support and numerous guidance at my research work as well as life in the academic years. Dr. Miao provides great freedom for my research, and was always willing to listen and share her experiences with me. Moreover, she often discussed with me about potential research directions over time, which inspired me in the area of agent-mediated interactive storytelling. I'd also like to thank my co-supervisor, Dr. Tan Ah-Hwee, who is most helpful by providing insightful comments and reviewing my papers. Moreover, special thanks go to Dr. Shen Zhiqi, for a lot of inspirations from him in the discussions and technical guidance. Thanks also go to Emerging Research Lab, School of Computer Engineering, Nanyang Technological University, for the financial support and computing facilities, and helpful lab technicians Wong Kia Yan and Thin Nandar Soe. Dr. Weng Jianshu gave me strong support, great advice and discreet work to improve my thesis. I also would like to thank other team members at developing the projects of serious games and interactive storytelling, who are Benny Tan,
Autonomous Agents & Multiagent Systems/Agent Theories, Architectures, and Languages, 2009
In this paper, we introduce an novel emotional agent system in 3D virtual world based on OCC (Ort... more In this paper, we introduce an novel emotional agent system in 3D virtual world based on OCC (Ortony, Clore and Collins) theory, FCM (Fuzzy Cognitive Map) and GoalNet. The agent system is designed based on Goal Net model. Emotional modeling and decision making are based on OCC and FCM inference. Emotions modeled by the OCC model are incorporated into FCM
2007 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Workshops, 2007
Personalization in virtual learning environments is the system ability to provide individualizati... more Personalization in virtual learning environments is the system ability to provide individualization and a set of personalized services such as personalized content management, learner model, or adaptive instant interaction. The intelligent agent technology has potential regarding the creation of such personalized, adaptive and interactive elearning applications. However, most of the available solutions have so far focused on porting existing courses with traditional teaching methods onto the virtual environments, making them available in an attractive animated interface without any fine-tuning and adaptation to the learner needs. This paper proposes a novel market-inspired collaboration model where the agents are self-interested autonomic elements collaborate to achieve a comprehensive learner model. Mentor agent makes decisions on top of a Dempster-Shafer belief accumulation to help student whenever she believes student has lost the clues and needs help. Proposed architecture is validated by applying on a sample agent augmented virtual environment designed to engage and motivate students at the lower secondary level in Singapore. Extensive experiments illustrate the effectiveness of the proposed interaction model where students have found the mentor agent as believable as a virtual teacher.
Proceedings of the 15th international conference on Intelligent user interfaces - IUI '10, 2010
Riding on the back of the rapid expansion of the Internet, online virtual worlds which combine th... more Riding on the back of the rapid expansion of the Internet, online virtual worlds which combine the prowess of interactive digital media and social networks have attained a high level of acceptance among the Net Generation users. This development prompted researchers to look into the potential of embedding learning contents into virtual worlds to create virtual learning environments (VLEs) that suit the need of the Net Generation learners. However, the special characteristics of virtual worlds that make them popular also pose great challenges to educators who wish to leverage their power. These challenges call for more sophisticated human computer interaction (HCI) mechanisms to assist learners to navigate the intriguing landscape of VLEs. In this paper, we demonstrate a teachable remembrance agent which acts as an intelligent user interface to provide innovative ways for students to interact with VLEs.
Lecture Notes in Computer Science, 2011
Interactive virtual storytelling attracts a lot of research interests to create engaging, interac... more Interactive virtual storytelling attracts a lot of research interests to create engaging, interactive and dynamic storytelling. However, currently there is a big gap between story plot modeling to believable character modeling and it is still a great challenge to create an interactive storytelling system from text-based stories for novice. In this paper, we propose an Agent-Oriented Methodology for Interactive Storytelling (AOMIS) to guide novice to generate interactive storytelling system. In this methodology, interactive storytelling system is constructed as a multi-agent system of director agent and character agents. The interactive storytelling is simulated as the goal pursuing of the agents affecting by user interactions and context changes. By viewing the interactive storytelling in the perspective of goal-oriented agents, our methodology is a hybrid of plot-oriented interactive storytelling and character-based interactive storytelling.
Lecture Notes in Computer Science, 2012
Virtual laboratory removes the barrier to conduct the experiments which are difficult to achieve ... more Virtual laboratory removes the barrier to conduct the experiments which are difficult to achieve in the real world. A lot of researches have been done on empowering the functionalities of virtual laboratories. However, it is still a challenge to provide a personalized learning experience in the virtual learning environment for the students. In our paper, we propose an agent-based virtual learning environment that helps the students to conduct the virtual experiments through the simulations and immersive role-playing. Agents are able to provide better instructions and stimulate the students conduct the deeper learning. Moreover, they monitor the learning process of the students, which is helpful to provide the customized learning in real-time. We use the methodology to teach the plant transportation for secondary school students in school and receives positive comments.
Virtual Actors in a Collaborative Story Creation System/9 António Brisson, Ana Paiva S-MADE: Interactive Storytelling Architecture through Goal Execution and Decomposition/17
Page 1. Contents Preface / vii Brian S. Magerko and Mark O. Riedl Rational Dialog in Interactive ... more Page 1. Contents Preface / vii Brian S. Magerko and Mark O. Riedl Rational Dialog in Interactive Games / 1 Maria Arinbjarnar Are We Telling the Same Story? Balancing Real and Virtual Actors in a Collaborative Story Creation System / 9 Ant��nio Brisson, Ana Paiva S-MADE: Interactive Storytelling Architecture through Goal Execution and Decomposition / 17 Yundong Cai, Zhiqi Shen, Chunyan Miao, Ah-Hwee Tan The Story Molecule: Narrative as Information / 21 Beth Cardier How Can We Be Serious in a Game? ...
Agent Mediated Interactive Storytelling in Virtual Environment
i-CAVE: Intelligent Agent-Augmented CAVE System
Cave Automatic Virtual Environment (CAVE) is a virtual reality environment with immersive 3D expe... more Cave Automatic Virtual Environment (CAVE) is a virtual reality environment with immersive 3D experience and engaging interactions, which would be a very good medium of entertainment, training or exercise for elderly. However, the raw CAVE system doesn’t provide semantic understanding of players’ behaviors and emotions in real-time, which leads to a challenge for application developers to create a personalized virtual environment for various players. In this paper we proposed an intelligent agentaugmented CAVE system, namely i-CAVE, to bridge the gap between application developers and end players, especially the elderly which has the special requirements of interactions. We have implemented the system with EON Studio in a Live CAVE environment and a case study called “Walking with Dinosaurs”. The interviewed players give very positive feedback on the immersion and engaging interactions. KeywordsCAVE, semantic information, agent
Cognitive Teachable Agent in Virtual Learning Environment with Fuzzy Cognitive Map
I would like to express my sincere appreciation to the following people for their support, encour... more I would like to express my sincere appreciation to the following people for their support, encouragement and help during my Ph.D study at Nanyang Technological University. First and foremost, I would like to take this opportunity to pay a high tribute to my PhD supervisor Dr. Miao Chun Yan, for her continuous support and numerous guidance at my research work as well as life in the academic years. Dr. Miao provides great freedom for my research, and was always willing to listen and share her experiences with me. Moreover, she often discussed with me about potential research directions over time, which inspired me in the area of agent-mediated interactive storytelling. I'd also like to thank my co-supervisor, Dr. Tan Ah-Hwee, who is most helpful by providing insightful comments and reviewing my papers. Moreover, special thanks go to Dr. Shen Zhiqi, for a lot of inspirations from him in the discussions and technical guidance. Thanks also go to Emerging Research Lab, School of Computer Engineering, Nanyang Technological University, for the financial support and computing facilities, and helpful lab technicians Wong Kia Yan and Thin Nandar Soe. Dr. Weng Jianshu gave me strong support, great advice and discreet work to improve my thesis. I also would like to thank other team members at developing the projects of serious games and interactive storytelling, who are Benny Tan,
Virtual laboratory removes the barrier to conduct the experiments which are difficult to achieve ... more Virtual laboratory removes the barrier to conduct the experiments which are difficult to achieve in the real world. A lot of researches have been done on empowering the functionalities of virtual laboratories. However, it is still a challenge to provide a personalized learning experience in the virtual learning environment for the students. In our paper, we propose an agent-based virtual learning environment that helps the students to conduct the virtual experiments through the simulations and immersive role-playing. Agents are able to provide better instructions and stimulate the students conduct the deeper learning. Moreover, they monitor the learning process of the students, which is helpful to provide the customized learning in real-time. We use the methodology to teach the plant transportation for secondary school students in school and receives positive comments.
Existing Parkinson's Disease (PD) diagnosis relies heavily on doctors' observations combined with... more Existing Parkinson's Disease (PD) diagnosis relies heavily on doctors' observations combined with neurological exams. Such a technique is often inconvenient, infrequent, and subjective, which leads to a high misdiagnosis rate. As several cardinal symptoms of PD require long term observations, a technology platform which allows potential PD patients to exhibit related behaviors in a natural setting over long period of time is needed. In this paper, we describe an agentbased game for the predictive diagnosis of PD. Agents in this tablet based game provide companionship, encouragement, and analysis capabilities to help retain users' interest, and analyze their risk of developing PD symptoms based on their in-game behavior. The game has been launched together with a world renowned PD research centre for clinical trial. It can potentially provide a new dimension of longitudinal interactive behavior data to assist early and more accurate PD diagnosis in the future.
Internet of Things Based Data Driven Storytelling for Supporting Social Connections
2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing, 2013
ABSTRACT The empty nest syndrome has become a significant modern social problem. Recent research ... more ABSTRACT The empty nest syndrome has become a significant modern social problem. Recent research has enhanced physical health monitoring for the elderly in smart home environment. However, the social needs of the empty nesters have been less supported in smart home designs. A compelling story about the elderly can make the adult children emotionally attached to their elderly parents and establish a healthy social connection between them. In this work, we propose a data-driven story generation system based on the Internet of Things (IoT). IoT based techniques enable the real time collection of the elderly's environment data through various sensors located in the home. The story generation system can weave the information mined from the sensor data into a story with high persuasive and emotion arousing power. Our system automatically collects raw sensor data on the cloud server and maps them to values of semantically meaningful context variables through activity functions. A director agent organizes the story plot with a Goal Net based approach and dynamically generates storylines based on a set of scene selection rules. The proposed system is carefully studied with a simulated smart home environment.
Technologies for E-Learning and Digital Entertainment
Interactive storytelling in the virtual environment attracts a lot of research interests in recen... more Interactive storytelling in the virtual environment attracts a lot of research interests in recent years. Story plot and character are two most important elements of a story. Based on these two elements, currently there are two research directions: plot-based and character-based interactive storytelling. However, plot-based approach lacks the refinement of character behaviors as character-based approach. On the other side, character-based approach does not follow a well organized story plot so that the moral of the story might be distorted. Therefore, there is a need to develop an integrated framework to achieve the balance between conveying story moral and enhancing the modeling of character behaviors. In this paper, we propose a hybrid system of the plot-based and character-based approaches. It is constructed as a multi-agent system (MAS), which involves a scriptwriter agent, a director agent, virtual actor agents and other support agents. Fuzzy Cognitive Goal Net (FCGN) is used by the scriptwriter agent to generate story plot, which includes various meaningful storylines. The director agent selects the storyline dynamically and dispatches the scenes to the virtual character agents as behaviors through the decomposing algorithms. With the system, dynamic storylines and character behaviors are generated in realtime based on the audience interactions and context changes. The audiences are able to experience different levels of interactions as an observer, a character or a director. A case study prototype called "Mystery Illness Investigation at Nanyang Town" has been implemented based on the proposed system.
2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence), 2008
To generate a believable and dynamic virtual world is a great challenge in interactive storytelli... more To generate a believable and dynamic virtual world is a great challenge in interactive storytelling. In this paper, we propose a model, namely Evolutionary Fuzzy Cognitive Map (E-FCM), to model the dynamic causal relationships among different context variables. As an extension to conventional FCM, E-FCM models not only the fuzzy causal relationships among the variables, but also the probabilistic property of causal relationships, and asynchronous activity update of the concepts. With this model, the context variables evolve in a dynamic and uncertain manner with the according evolving time. As a result, the virtual world is presented more realistically and dynamically.
Lecture Notes in Computer Science, 2012
As treasure houses of human history and knowledge with rich repositories on cultural heritage, mu... more As treasure houses of human history and knowledge with rich repositories on cultural heritage, museums and libraries have advantages of presence experience and powerful searching abilities respectively. We propose in this paper an integrated design of virtual museum and digital library in the preservation of cultural heritage of the Chinese "qipao", as an example of preservation of cultural heritage resources in digital space.
Crowdsourcing Metadata Schema Generation for Chinese-Style Costume Digital Library
Lecture Notes in Computer Science, 2012
Digital library (DL) has been used as a very important tool for cultural preservation and appreci... more Digital library (DL) has been used as a very important tool for cultural preservation and appreciation. However, it is a great challenge to generate effective and efficient metadata schemas to describe digital artefacts once common metadata schema standards are not enough. In this paper, we propose a crowdsourcing metadata schema generation methodology and use it to create a new metadata schema, "Costume Core" (CC) for Chinese-style costume-related heritage resources, which could be used by other costume digital libraries. We then conducted an initial survey to get users’ feedback on the ranked importance of the metadata elements in the CC schema.