Chia Shen - Academia.edu (original) (raw)

Papers by Chia Shen

With the availability of new input devices such as DiamondTouch [1], multiple users can now inter... more With the availability of new input devices such as DiamondTouch [1], multiple users can now interact with a 2D surface using both hands and/or multiple fingers simultaneously. From our experience in working on multiple preferred orientations of documents in an augmented tabletop environment [2,3], we noticed that the orientation of documents on the table and the number of users are not the only entities that naturally present multiplicity and concurrency. In this paper we present our vision on how users can use more than one fingers to collaboratively manipulate multiple documents in parallel. Research Vision Multiple mice input methods and the effect of multiple mice for supporting co-located collaboration have been previously investigated in the context of Mmouse [maryland Mmouse] and SDG (Single Display Groupware) [4,5]. These studies were all based on systems using a vertical desktop display. It is common practice for multiple people working at a physical horizontal table to pas...

Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 2015

Interactive surfaces are increasingly common in museums and other informal learning environments ... more Interactive surfaces are increasingly common in museums and other informal learning environments where they are seen as a medium for promoting social engagement. However, despite their increasing prevalence, we know very little about factors that contribute to collaboration and learning around interactive surfaces. In this paper we present analyses of visitor engagement around several multi-touch tabletop science exhibits. Observations of 629 visitors were collected through two widely used techniques: video study and shadowing. We make four contributions: 1) we present an algorithm for identifying groups within a dynamic flow of visitors through an exhibit hall; 2) we present measures of group-level engagement along with methods for statistically analyzing these measures; 3) we assess the effect of observational techniques on visitors' engagement, demonstrating that consented video studies do not necessarily reflect visitor behavior in more naturalistic circumstances; and 4) we present an analysis showing that groups of two, groups with both children and adults, and groups that take turns spend longer at the exhibits and engage more with scientific concepts.

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2009

We present CThru, a self-guided video-based educational environment in a large multi-display sett... more We present CThru, a self-guided video-based educational environment in a large multi-display setting. We employ a video-centered approach, creating and combining multimedia contents of different formats with a story-telling education video. With the support of new display form factors in the environment, viewing a sequential educational video thread is replaced by the immersive learning experience of hands-on exploration and manipulation in a multi-dimensional information space. We demonstrate CThru with an animation clip in cellular biology, supplementing visible objects in the video with rich domain-specific multimedia information and interactive 3D models. We describe CThru's design rationale and implementation. We also discuss a pilot study and what it revealed with respect to CThru's interface and the usage pattern of the tabletop and the associated large wall display.

International Journal of Computer-Supported Collaborative Learning, 2015

In this paper we present a qualitative analysis of natural history museum visitor interaction aro... more In this paper we present a qualitative analysis of natural history museum visitor interaction around a multi-touch tabletop exhibit called DeepTree that we designed around concepts of evolution and common descent. DeepTree combines several large scientific datasets and an innovative visualization technique to display a phylogenetic tree of life consisting of over 70,000 species. After describing our design, we present a study involving pairs of children interacting with DeepTree in two natural history museums. Our analysis focuses on two questions. First, how do dyads negotiate their moment-to-moment exploration of the exhibit? Second, how do dyads develop and negotiate their understanding of evolutionary concepts? In order to address these questions we present an analytical framework that describes dyads' exploration along two dimensions: coordination and target of action. This framework reveals four distinct patterns of interaction, which, we argue, are relevant for similar interactive designs. We conclude with a discussion of the role of design in helping visitors make sense of interactive experiences involving the visualization of large scientific datasets.

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2007

Co-located collaborators often work over physical tabletops using combinations of expressive hand... more Co-located collaborators often work over physical tabletops using combinations of expressive hand gestures and verbal utterances. This paper provides the first observations of how pairs of people communicated and interacted in a multimodal digital table environment built atop existing single user applications. We contribute to the understanding of these environments in two ways. First, we saw that speech and gesture commands served double duty as both commands to the computer, and as implicit communication to others. Second, in spite of limitations imposed by the underlying single-user application, people were able to work together simultaneously, and they performed interleaving acts: the graceful mixing of inter-person speech and gesture actions as commands to the system. This work contributes to the intricate understanding of multiuser multimodal digital table interaction.

Proceedings of the 2002 ACM conference on Computer supported cooperative work, 2002

Organizations, families, institutions evolve a shared culture and history. In this work, we descr... more Organizations, families, institutions evolve a shared culture and history. In this work, we describe a system to facilitate conversation and storytelling about this collective past. Users explore digital archives of shared materials such as photographs, video, and text documents on a tabletop interface. Both the software and the interface encourage natural conversation and reflection. This work is an application of our ongoing research on systems for multiple, co-present users to explore digital collections. In this paper, we present a case study of our own group history along with the software extensions developed for this scenario. These extensions include methods for easily branching off from and returning to previous threads of the exploration, incorporating background contexts that support a variety of view points and flexible story sharing, and supporting the active and passive discovery of relevant information.

CHI '05 Extended Abstracts on Human Factors in Computing Systems, 2005

We describe a technique that supports the previewing of navigation, exploration, and editing oper... more We describe a technique that supports the previewing of navigation, exploration, and editing operations by providing convenient Undo for unsuccessful and/or undesirable actions on multi-level input devices such as touch screens and pen-based computers. By adding a Glimpse state to traditional three-state pressure sensitive input devices, users are able to preview the effects of their editing without committing to them. From this Glimpse state, users can undo their action as easily as they can commit to it, making Glimpse most appropriate for systems in which the user is likely to try out many variations of an edit before finding the right one. Exploration is encouraged as the cumbersome returning to a menu or keyboard to issue an Undo command is eliminated. Glimpse has the added benefits that the negative effects of inconsistencies in the Undo feature within an application are reduced.

Graphics Interface, 2005

We present two experiments on the use of non-speech audio at an interactive multi-touch, multi-us... more We present two experiments on the use of non-speech audio at an interactive multi-touch, multi-user tabletop display. We first investigate the use of two categories of reactive auditory feedback: affirmative sounds that confirm user actions and negative sounds that indicate errors. Our results show that affirmative auditory feedback may improve one's awareness of group activity at the expense of one's

Proceedings of the 25th annual ACM symposium on User interface software and technology, 2012

Multi-touch technology lends itself to collaborative crowd interaction (CI). However, common tap-... more Multi-touch technology lends itself to collaborative crowd interaction (CI). However, common tap-operated widgets are impractical for CI, since they are susceptible to accidental touches and interference from other users. We present a novel multi-touch interface called FlowBlocks in which every UI action is invoked through a small sequence of user actions: dragging parametric UI-Blocks, and dropping them over operational UI-Docks. The FlowBlocks approach is advantageous for CI because it a) makes accidental touches inconsequential; and b) introduces design parameters for mutual awareness, concurrent input, and conflict management. FlowBlocks was successfully used on the floor of a busy natural history museum. We present the complete design space and describe a year-long iterative design and evaluation process which employed the Rapid Iterative Test and Evaluation (RITE) method in a museum setting.

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2012

In this paper we describe visitor interaction with an interactive tabletop exhibit on evolution t... more In this paper we describe visitor interaction with an interactive tabletop exhibit on evolution that we designed for use in natural history museums. We video recorded 30 families using the exhibit at the Harvard Museum of Natural History. We also observed an additional 50 social groups interacting with the exhibit without video recording. The goal of this research is to explore ways to develop "successful" interactive tabletop exhibits for museums. To determine criteria for success in this context, we borrow the concept of Active Prolonged Engagement (APE) from the science museum literature. Research on APE sets a high standard for visitor engagement and learning, and it offers a number of useful concepts and measures for research on interactive surfaces in the wild. In this paper we adapt and expand on these measures and apply them to our tabletop exhibit. Our results show that visitor groups collaborated effectively and engaged in focused, on-topic discussion for prolonged periods of time. To understand these results, we analyze visitor conversation at the exhibit. Our analysis suggests that social practices of game play contributed substantially to visitor collaboration and engagement with the exhibit.

Interactions, 2003

Recent advances in technology have made it possible to share digital contents.

2008 3rd IEEE International Workshop on Horizontal Interactive Human Computer Systems, 2008

The WeSpace is a long-term project dedicated to the creation of environments supporting walkup an... more The WeSpace is a long-term project dedicated to the creation of environments supporting walkup and share collaboration among small groups. The focus of our system design has been to provide 1) groups with mechanisms to easily share their own data and 2) necessary native visual applications suitable on large display environments. Our current prototype system includes both a large high-resolution data wall and an interactive table. These are utilized to provide a focal point for collaborative interaction with data and applications. In this paper, we describe in detail the designs behind the current prototype system. In particular, we present 1) the infrastructure which allows users to connect and visually share their laptop content on-the-fly, and supports the extension of native visualization applications, and 2) the table-centric design employed in customized WeSpace applications to support cross-surface interactions. We will also describe elements of our user-centered iterative design process, in particular the results from a late-stages session which saw our astrophysicist participants successfully use the WeSpace to collaborate on their own real research problem.

Proceedings of the 8th international conference on Multimodal interfaces, 2006

Most commercial software applications are designed for a single user using a keyboard/mouse over ... more Most commercial software applications are designed for a single user using a keyboard/mouse over an upright monitor. Our interest is exploiting these systems so they work over a digital table. Mirroring what people do when working over traditional tables, we want to allow multiple people to interact naturally with the tabletop application and with each other via rich speech and hand gestures. In previous papers, we illustrated multiuser gesture and speech interaction on a digital table for geospatial applications-Google Earth, Warcraft III and The Sims. In this paper, we describe our underlying architecture: GSI DEMO. First, GSI DEMO creates a run-time wrapper around existing single user applications: it accepts and translates speech and gestures from multiple people into a single stream of keyboard and mouse inputs recognized by the application. Second, it lets people use multimodal demonstration-instead of programming-to quickly map their own speech and gestures to these keyboard/mouse inputs. For example, continuous gestures are trained by saying "Computer, when I do [one finger gesture], you do [mouse drag]". Similarly, discrete speech commands can be trained by saying "Computer, when I say [layer bars], you do [keyboard and mouse macro]". The end result is that end users can rapidly transform single user commercial applications into a multiuser , multimodal digital tabletop system.

Proceedings of the 2004 ACM conference on Computer supported cooperative work, 2004

The status quo for co-located groupware is to assume that "social protocols" (standards of polite... more The status quo for co-located groupware is to assume that "social protocols" (standards of polite behavior) are sufficient to coordinate the actions of a group of users; however, prior studies of groupware use as well as our own observations of groups using a shared tabletop display suggest potential for improving groupware interfaces by incorporating coordination policies-direct manipulation mechanisms for avoiding and resolving conflicts. We discuss our observations of group tabletop usage and present our coordination framework. We conclude with example usage scenarios and discuss future research suggested by this framework.

CHI '04 Extended Abstracts on Human Factors in Computing Systems, 2004

Group work frequently involves transitions between periods of active collaboration and periods of... more Group work frequently involves transitions between periods of active collaboration and periods of individual activity. We aim to support this typical work practice by introducing four tabletop direct-manipulation interaction techniques that can be used to transition the status of an electronic document from private to group-accessible. After presenting our four techniques-release, relocate, reorient, and resize-we discuss the results of an empirical study that compares and evaluates these mechanisms for sharing documents in a co-located tabletop environment.

CHI '05 Extended Abstracts on Human Factors in Computing Systems, 2005

We present a new interaction technique for large direct-touch displays called Modal Spaces. Modal... more We present a new interaction technique for large direct-touch displays called Modal Spaces. Modal interfaces require the user to keep track of the state of the system. The Modal Spaces technique adds screen location aas an additional parameter of the interaction. Each modal region on the display supports a particular set of input actions and the visual background indicates the space's use. This "workbech approach" exploits the larger form factor of display. Our spatial multiplexing of the display supports a document-centric paradigm (as opposed to application-centric), enabling input gesture reuse, while complementing and enhancing the current existing practices of modal interfaces. We present a proof-of-concept system and discuss porential applications, design issues, and future research directions.

Recent advances in technology have made it possible to share digital contents.

International Journal of Computer Vision, 2004

We present a user-centric system for visualization and layout for content-based image retrieval. ... more We present a user-centric system for visualization and layout for content-based image retrieval. Image features (visual and/or semantic) are used to display retrievals as thumbnails in a 2-D spatial layout or "configuration" which conveys all pair-wise mutual similarities. A graphical optimization technique is used to provide maximally uncluttered and informative layouts. Moreover, a novel subspace feature weighting technique can be used to modify 2-D layouts in a variety of context-dependent ways. An efficient computational technique for subspace weighting and re-estimation leads to a simple user-modeling framework whereby the system can learn to display query results based on layout examples (or relevance feedback) provided by the user. The resulting retrieval, browsing and visualization can adapt to the user's (time-varying) notions of content, context and preferences in style and interactive navigation. Monte Carlo simulations with machine-generated layouts as well as pilot user studies have demonstrated the ability of this framework to model or "mimic" users, by automatically generating layouts according to their preferences

IEEE Transactions on Visualization and Computer Graphics, 2012

Fig. 1. The DeepTree Exhibit: main view of the tree of life, image reel DeepTree (middle). Specia... more Fig. 1. The DeepTree Exhibit: main view of the tree of life, image reel DeepTree (middle). Special learning activity for common descent, inheritance

With the availability of new input devices such as DiamondTouch [1], multiple users can now inter... more With the availability of new input devices such as DiamondTouch [1], multiple users can now interact with a 2D surface using both hands and/or multiple fingers simultaneously. From our experience in working on multiple preferred orientations of documents in an augmented tabletop environment [2,3], we noticed that the orientation of documents on the table and the number of users are not the only entities that naturally present multiplicity and concurrency. In this paper we present our vision on how users can use more than one fingers to collaboratively manipulate multiple documents in parallel. Research Vision Multiple mice input methods and the effect of multiple mice for supporting co-located collaboration have been previously investigated in the context of Mmouse [maryland Mmouse] and SDG (Single Display Groupware) [4,5]. These studies were all based on systems using a vertical desktop display. It is common practice for multiple people working at a physical horizontal table to pas...

Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 2015

Interactive surfaces are increasingly common in museums and other informal learning environments ... more Interactive surfaces are increasingly common in museums and other informal learning environments where they are seen as a medium for promoting social engagement. However, despite their increasing prevalence, we know very little about factors that contribute to collaboration and learning around interactive surfaces. In this paper we present analyses of visitor engagement around several multi-touch tabletop science exhibits. Observations of 629 visitors were collected through two widely used techniques: video study and shadowing. We make four contributions: 1) we present an algorithm for identifying groups within a dynamic flow of visitors through an exhibit hall; 2) we present measures of group-level engagement along with methods for statistically analyzing these measures; 3) we assess the effect of observational techniques on visitors' engagement, demonstrating that consented video studies do not necessarily reflect visitor behavior in more naturalistic circumstances; and 4) we present an analysis showing that groups of two, groups with both children and adults, and groups that take turns spend longer at the exhibits and engage more with scientific concepts.

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2009

We present CThru, a self-guided video-based educational environment in a large multi-display sett... more We present CThru, a self-guided video-based educational environment in a large multi-display setting. We employ a video-centered approach, creating and combining multimedia contents of different formats with a story-telling education video. With the support of new display form factors in the environment, viewing a sequential educational video thread is replaced by the immersive learning experience of hands-on exploration and manipulation in a multi-dimensional information space. We demonstrate CThru with an animation clip in cellular biology, supplementing visible objects in the video with rich domain-specific multimedia information and interactive 3D models. We describe CThru's design rationale and implementation. We also discuss a pilot study and what it revealed with respect to CThru's interface and the usage pattern of the tabletop and the associated large wall display.

International Journal of Computer-Supported Collaborative Learning, 2015

In this paper we present a qualitative analysis of natural history museum visitor interaction aro... more In this paper we present a qualitative analysis of natural history museum visitor interaction around a multi-touch tabletop exhibit called DeepTree that we designed around concepts of evolution and common descent. DeepTree combines several large scientific datasets and an innovative visualization technique to display a phylogenetic tree of life consisting of over 70,000 species. After describing our design, we present a study involving pairs of children interacting with DeepTree in two natural history museums. Our analysis focuses on two questions. First, how do dyads negotiate their moment-to-moment exploration of the exhibit? Second, how do dyads develop and negotiate their understanding of evolutionary concepts? In order to address these questions we present an analytical framework that describes dyads' exploration along two dimensions: coordination and target of action. This framework reveals four distinct patterns of interaction, which, we argue, are relevant for similar interactive designs. We conclude with a discussion of the role of design in helping visitors make sense of interactive experiences involving the visualization of large scientific datasets.

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2007

Co-located collaborators often work over physical tabletops using combinations of expressive hand... more Co-located collaborators often work over physical tabletops using combinations of expressive hand gestures and verbal utterances. This paper provides the first observations of how pairs of people communicated and interacted in a multimodal digital table environment built atop existing single user applications. We contribute to the understanding of these environments in two ways. First, we saw that speech and gesture commands served double duty as both commands to the computer, and as implicit communication to others. Second, in spite of limitations imposed by the underlying single-user application, people were able to work together simultaneously, and they performed interleaving acts: the graceful mixing of inter-person speech and gesture actions as commands to the system. This work contributes to the intricate understanding of multiuser multimodal digital table interaction.

Proceedings of the 2002 ACM conference on Computer supported cooperative work, 2002

Organizations, families, institutions evolve a shared culture and history. In this work, we descr... more Organizations, families, institutions evolve a shared culture and history. In this work, we describe a system to facilitate conversation and storytelling about this collective past. Users explore digital archives of shared materials such as photographs, video, and text documents on a tabletop interface. Both the software and the interface encourage natural conversation and reflection. This work is an application of our ongoing research on systems for multiple, co-present users to explore digital collections. In this paper, we present a case study of our own group history along with the software extensions developed for this scenario. These extensions include methods for easily branching off from and returning to previous threads of the exploration, incorporating background contexts that support a variety of view points and flexible story sharing, and supporting the active and passive discovery of relevant information.

CHI '05 Extended Abstracts on Human Factors in Computing Systems, 2005

We describe a technique that supports the previewing of navigation, exploration, and editing oper... more We describe a technique that supports the previewing of navigation, exploration, and editing operations by providing convenient Undo for unsuccessful and/or undesirable actions on multi-level input devices such as touch screens and pen-based computers. By adding a Glimpse state to traditional three-state pressure sensitive input devices, users are able to preview the effects of their editing without committing to them. From this Glimpse state, users can undo their action as easily as they can commit to it, making Glimpse most appropriate for systems in which the user is likely to try out many variations of an edit before finding the right one. Exploration is encouraged as the cumbersome returning to a menu or keyboard to issue an Undo command is eliminated. Glimpse has the added benefits that the negative effects of inconsistencies in the Undo feature within an application are reduced.

Graphics Interface, 2005

We present two experiments on the use of non-speech audio at an interactive multi-touch, multi-us... more We present two experiments on the use of non-speech audio at an interactive multi-touch, multi-user tabletop display. We first investigate the use of two categories of reactive auditory feedback: affirmative sounds that confirm user actions and negative sounds that indicate errors. Our results show that affirmative auditory feedback may improve one's awareness of group activity at the expense of one's

Proceedings of the 25th annual ACM symposium on User interface software and technology, 2012

Multi-touch technology lends itself to collaborative crowd interaction (CI). However, common tap-... more Multi-touch technology lends itself to collaborative crowd interaction (CI). However, common tap-operated widgets are impractical for CI, since they are susceptible to accidental touches and interference from other users. We present a novel multi-touch interface called FlowBlocks in which every UI action is invoked through a small sequence of user actions: dragging parametric UI-Blocks, and dropping them over operational UI-Docks. The FlowBlocks approach is advantageous for CI because it a) makes accidental touches inconsequential; and b) introduces design parameters for mutual awareness, concurrent input, and conflict management. FlowBlocks was successfully used on the floor of a busy natural history museum. We present the complete design space and describe a year-long iterative design and evaluation process which employed the Rapid Iterative Test and Evaluation (RITE) method in a museum setting.

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2012

In this paper we describe visitor interaction with an interactive tabletop exhibit on evolution t... more In this paper we describe visitor interaction with an interactive tabletop exhibit on evolution that we designed for use in natural history museums. We video recorded 30 families using the exhibit at the Harvard Museum of Natural History. We also observed an additional 50 social groups interacting with the exhibit without video recording. The goal of this research is to explore ways to develop "successful" interactive tabletop exhibits for museums. To determine criteria for success in this context, we borrow the concept of Active Prolonged Engagement (APE) from the science museum literature. Research on APE sets a high standard for visitor engagement and learning, and it offers a number of useful concepts and measures for research on interactive surfaces in the wild. In this paper we adapt and expand on these measures and apply them to our tabletop exhibit. Our results show that visitor groups collaborated effectively and engaged in focused, on-topic discussion for prolonged periods of time. To understand these results, we analyze visitor conversation at the exhibit. Our analysis suggests that social practices of game play contributed substantially to visitor collaboration and engagement with the exhibit.

Interactions, 2003

Recent advances in technology have made it possible to share digital contents.

2008 3rd IEEE International Workshop on Horizontal Interactive Human Computer Systems, 2008

The WeSpace is a long-term project dedicated to the creation of environments supporting walkup an... more The WeSpace is a long-term project dedicated to the creation of environments supporting walkup and share collaboration among small groups. The focus of our system design has been to provide 1) groups with mechanisms to easily share their own data and 2) necessary native visual applications suitable on large display environments. Our current prototype system includes both a large high-resolution data wall and an interactive table. These are utilized to provide a focal point for collaborative interaction with data and applications. In this paper, we describe in detail the designs behind the current prototype system. In particular, we present 1) the infrastructure which allows users to connect and visually share their laptop content on-the-fly, and supports the extension of native visualization applications, and 2) the table-centric design employed in customized WeSpace applications to support cross-surface interactions. We will also describe elements of our user-centered iterative design process, in particular the results from a late-stages session which saw our astrophysicist participants successfully use the WeSpace to collaborate on their own real research problem.

Proceedings of the 8th international conference on Multimodal interfaces, 2006

Most commercial software applications are designed for a single user using a keyboard/mouse over ... more Most commercial software applications are designed for a single user using a keyboard/mouse over an upright monitor. Our interest is exploiting these systems so they work over a digital table. Mirroring what people do when working over traditional tables, we want to allow multiple people to interact naturally with the tabletop application and with each other via rich speech and hand gestures. In previous papers, we illustrated multiuser gesture and speech interaction on a digital table for geospatial applications-Google Earth, Warcraft III and The Sims. In this paper, we describe our underlying architecture: GSI DEMO. First, GSI DEMO creates a run-time wrapper around existing single user applications: it accepts and translates speech and gestures from multiple people into a single stream of keyboard and mouse inputs recognized by the application. Second, it lets people use multimodal demonstration-instead of programming-to quickly map their own speech and gestures to these keyboard/mouse inputs. For example, continuous gestures are trained by saying "Computer, when I do [one finger gesture], you do [mouse drag]". Similarly, discrete speech commands can be trained by saying "Computer, when I say [layer bars], you do [keyboard and mouse macro]". The end result is that end users can rapidly transform single user commercial applications into a multiuser , multimodal digital tabletop system.

Proceedings of the 2004 ACM conference on Computer supported cooperative work, 2004

The status quo for co-located groupware is to assume that "social protocols" (standards of polite... more The status quo for co-located groupware is to assume that "social protocols" (standards of polite behavior) are sufficient to coordinate the actions of a group of users; however, prior studies of groupware use as well as our own observations of groups using a shared tabletop display suggest potential for improving groupware interfaces by incorporating coordination policies-direct manipulation mechanisms for avoiding and resolving conflicts. We discuss our observations of group tabletop usage and present our coordination framework. We conclude with example usage scenarios and discuss future research suggested by this framework.

CHI '04 Extended Abstracts on Human Factors in Computing Systems, 2004

Group work frequently involves transitions between periods of active collaboration and periods of... more Group work frequently involves transitions between periods of active collaboration and periods of individual activity. We aim to support this typical work practice by introducing four tabletop direct-manipulation interaction techniques that can be used to transition the status of an electronic document from private to group-accessible. After presenting our four techniques-release, relocate, reorient, and resize-we discuss the results of an empirical study that compares and evaluates these mechanisms for sharing documents in a co-located tabletop environment.

CHI '05 Extended Abstracts on Human Factors in Computing Systems, 2005

We present a new interaction technique for large direct-touch displays called Modal Spaces. Modal... more We present a new interaction technique for large direct-touch displays called Modal Spaces. Modal interfaces require the user to keep track of the state of the system. The Modal Spaces technique adds screen location aas an additional parameter of the interaction. Each modal region on the display supports a particular set of input actions and the visual background indicates the space's use. This "workbech approach" exploits the larger form factor of display. Our spatial multiplexing of the display supports a document-centric paradigm (as opposed to application-centric), enabling input gesture reuse, while complementing and enhancing the current existing practices of modal interfaces. We present a proof-of-concept system and discuss porential applications, design issues, and future research directions.

Recent advances in technology have made it possible to share digital contents.

International Journal of Computer Vision, 2004

We present a user-centric system for visualization and layout for content-based image retrieval. ... more We present a user-centric system for visualization and layout for content-based image retrieval. Image features (visual and/or semantic) are used to display retrievals as thumbnails in a 2-D spatial layout or "configuration" which conveys all pair-wise mutual similarities. A graphical optimization technique is used to provide maximally uncluttered and informative layouts. Moreover, a novel subspace feature weighting technique can be used to modify 2-D layouts in a variety of context-dependent ways. An efficient computational technique for subspace weighting and re-estimation leads to a simple user-modeling framework whereby the system can learn to display query results based on layout examples (or relevance feedback) provided by the user. The resulting retrieval, browsing and visualization can adapt to the user's (time-varying) notions of content, context and preferences in style and interactive navigation. Monte Carlo simulations with machine-generated layouts as well as pilot user studies have demonstrated the ability of this framework to model or "mimic" users, by automatically generating layouts according to their preferences

IEEE Transactions on Visualization and Computer Graphics, 2012

Fig. 1. The DeepTree Exhibit: main view of the tree of life, image reel DeepTree (middle). Specia... more Fig. 1. The DeepTree Exhibit: main view of the tree of life, image reel DeepTree (middle). Special learning activity for common descent, inheritance