TEI (Tangible, Embedded, and Embodied Interaction) Research Papers (original) (raw)

This article presents a paper-based Tangible User Interface (TUI) that facilitates the production of complex queries on a Cultural Heritage (CH) repository. The system helps to easily make use of the data elements and Boolean logic that... more

This article presents a paper-based Tangible User Interface (TUI) that facilitates the production of complex queries on a Cultural Heritage (CH) repository. The system helps to easily make use of the data elements and Boolean logic that describe the collections. This research presents a design methodology divided into two main phases: A User Experience (UX) and User Centred Design (UCD) where potential users’ behaviours are analysed, followed by the development and evaluation of the TUI prototype.
The TUI uses off the shelf electronics and a paper-based set of tokens to engage the user with the system, thus facilitating the exploration with CH collections through querying.

We present ArtVis, an advanced user interface combining state-of-the-art visualisation techniques and tangible interaction to explore the Web Gallery of Art digital artwork collection consisting of almost 28000 European artworks. The... more

We present ArtVis, an advanced user interface combining state-of-the-art visualisation techniques and tangible interaction to explore the Web Gallery of Art digital artwork collection consisting of almost 28000 European artworks. The graphical ArtVis interface contains three separate visualisation panels that allow users to temporally, semantically and geographically investigate the digital artwork collection. The goal of ArtVis is that users can gain new insights by visually analysing and exploring a large data set rather than to only provide detailed information about individual artworks. This explorative ArtVis interaction style is further fostered by a tangible user interface where physical artefacts can be used as simple handles to drive the faceted browsing of the rich digital information space.

The boundaries between “the digital” and our everyday physical world are dissolving as we develop more physical ways of interacting with computing. This forum presents some of the topics discussed in the colorful multidisciplinary field... more

The boundaries between “the digital” and our everyday physical world are dissolving as we develop more physical ways of interacting with computing. This forum presents some of the topics discussed in the colorful multidisciplinary field of tangible and embodied interaction.

We report the results of an extended empirical two-stage study on the aesthetics of hybrid objects that combine form and behaviour. By combining two shapes (spheres and cubes); two sizes (7.5cm and 15cm); two materials (fabric and... more

We report the results of an extended empirical two-stage study on the aesthetics of hybrid objects that combine form and behaviour. By combining two shapes (spheres and cubes); two sizes (7.5cm and 15cm); two materials (fabric and plastic); and four behaviours (emitting light, emitting sound, vibrating or displaying no behaviour) we created 32 objects that differ for a single feature. In a between-participants study, 175 participants assessed and described the 32 objects. From this, seven dimensions were identified: pleasant; interesting; comfortable; playful; relaxing; special and surprising. In a second between-participants experiment 486 participants rated each object on the seven dimensions from the first study. Overall Spheres, Fabric, and Vibration were the preferred features, but for some of the dimensions specific combinations of features were rated more positively. This paper contribution is twofold: it provides a first study on the aesthetic of tangible interaction as a combination of form and behaviour outlining a potential instrument to measure it; and it provides empirical evidence of the value of experimenting with different forms (spheres) and material (fabric) even if they are difficult to create as they generate the strongest aesthetic effects.

This paper explores the potential of tangible and embodied interaction for encouraging a multisensory engagement with museum objects and artefacts on display, by means of focusing on the subtleties of devising and planning for evaluation... more

This paper explores the potential of tangible and embodied interaction for encouraging a multisensory engagement with museum objects and artefacts on display, by means of focusing on the subtleties of devising and planning for evaluation and audience research. Measuring the impact of new technologies is one of the main challenges identified in the 2015 NMC Horizon report (Museum Edition). The challenge is even greater for emerging concepts, technologies, and approaches, such as the use of tangible and embodied interaction in museums and other Cultural Heritage settings. Taking as an example two case-studies from the EU meSch project, from Museon and Allard Pierson Museum in the Netherlands, we discuss our plan for devising and carrying out audience research so as to " document, " analyse, and interpret the impact of digitally enhanced, tangible, embodied, and multisensory museum visiting experiences. Our intention is to provide an honest account of the different strengths and weaknesses encountered for all evaluation methodologies that were used, namely observations, interviews, video data, questionnaires, meaning maps, and post-visit interviews. We also share and discuss lessons learned, insights and best practices that could be of benefit for museum and audience research professionals.

Touch-based interactions with computing technologies have become commonplace in the last few years, from mobile phones to tabletop surfaces. The sense of touch however is not limited to the hand; the entire skin surface of the body is... more

Touch-based interactions with computing technologies have become commonplace in the last few years, from mobile phones to tabletop surfaces. The sense of touch however is not limited to the hand; the entire skin surface of the body is available for tactile interaction. In architecture, researchers are now investigating the potential of interactive surfaces for future architectonic elements, such as walls, floors and ceilings. Apart from the traditional focus on the visual and spatial design considerations of such elements, tactile interaction with interactive surfaces is of growing interest. We present an interactive folded surface as a prototype of future interactive architectural surfaces. We explain how physiological understandings of touch and tactile interaction informed the design choices of the prototype. Our work contributes to understandings of how the material properties and interactive behaviours of these new surfaces will afford new kinds of human experience centred on the sense of touch.

Man, even though he may sometimes dream of being a pure spirit free from all material constraints, actually exists as a body in a world which is itself essentially constructed as a spatialized system of bodies; and in this constructed... more

Man, even though he may sometimes dream of being a pure spirit free from all material constraints, actually exists as a body in a world which is itself essentially constructed as a spatialized system of bodies; and in this constructed world features such as resistance, impenetrability, inertia, weight and forces are not accidents or exceptions but the rule.
The reflexions presented in this work aim at elucidating on a phenomenological level the relation that man entertains with the tangible aspects of his environment, and the function that this relation plays in the construction of the different sectors of his existence. The first of the sectors is perception: the opening on an ambient world that is pregnant with organisation and meaning. On the one hand, we shall take into account and conceptualize the way in which the tangibility of the world (the testing experience that the individual can have of resistance of his environment in the frame of a direct bodily relation with that environment, but more generally the understanding that the individual possesses of the very possibility of such a relation) participates in setting up the rationality by which man renders his world intelligible. On the other hand, we shall identify the structures of “subjectivity” (to speak the language of psychology, the structures of cognition) which make possible the specific way in which man constructs his experience and understanding of tangible resistance – whether this resistance is perceived in the context of an actual bodily engagement, or whether it is envisaged “indirectly” in the guise of a “simple” possibility.
We shall show here, on the basis of phenomenological analyses as well as elements coming from empirical psychology (experimental psychology as well as neuropsychology), that the relation that man has with the resistance of his ambient world depends on an opening towards the possible; and that the organisation of the ambient world in the form of a space which can contain material structures, or structures capable of opposing a resistance to the body, depends on a rationality which consists of making phenomena intelligible by interpreting them with reference to those capacities for action and for passion which the body confers on us, to the power that the body provides us with and the constraints to which it submits us.
Thus, even though the realm of material things – tangible objects, bodies – functions as an archetype of “presence” and of “being” (indeed, what is more “real” than a body?), in the last resort it gains its phenomenal character from the fact that it crystallizes for the person who perceives it a stream of virtual possibilities; hence, the here-and-now presence of the “tangible world” is in an essential way indebted to the realm of that which is not actually realized.

HAID'12. Even though multisensory environments (MSE) incorporate artifacts and technology to provide sensory stimuli, most of these artifacts are non-interactive. Twenty-four children with profound developmental disabilities from three... more

HAID'12.
Even though multisensory environments (MSE) incorporate artifacts and technology to provide sensory stimuli, most of these artifacts are non-interactive. Twenty-four children with profound developmental disabilities from three MSE institutions have been involved in a research study. A handful of interactive design artifacts, which have been developed as a tool for ideation and to enhance the use of MSE by promoting children’s engagement are presented. With these artifacts the children have shown us a vast topology of interaction and bodily engagement, showing a potential for haptic and audio interactive design fields to contribute to a more participatory MSE practice.

The last two decades have seen the emergence and steady development of tangible user interfaces. While most of these interfaces are applied for input—with output still on traditional computer screens—the goal of programmable matter and... more

The last two decades have seen the emergence and steady development of tangible user interfaces. While most of these interfaces are applied for input—with output still on traditional computer screens—the goal of programmable matter and actuated shape-changing materials is to directly use the physical objects for visual or tangible feedback. Advances in material sciences and flexible display technologies are investigated to enable such reconfigurable physical objects. While existing solutions aim for making physical objects more controllable via the digital world, we propose an approach where holograms (virtual objects) in a mixed reality environment are augmented with physical variables such as shape, texture or temperature. As such, the support for mobility forms an important contribution of the proposed solution since it enables users to freely move within and across environments. Furthermore, our augmented virtual objects can co-exist in a single environment with programmable matter and other actuated shape-changing solutions. The future potential of the proposed approach is illustrated in two usage scenarios and we hope that the presentation of our work in progress on a novel way to realise tangible holograms will foster some lively discussions in the CHI community.

This research explores design opportunities where tangible interaction enables new ways to engage visitors with the stories and artefacts on display, not in a museum as such, but within a house museum – a particular type of heritage site... more

This research explores design opportunities where tangible interaction enables new ways to engage visitors with the stories and artefacts on display, not in a museum as such, but within a house museum – a particular type of heritage site where I noticed little attention from the field of interaction design. The work sits between the fields of design (e.g. exhibition design), heritage, and technology (e.g. HCI) and it investigates how the approach to designing for house museums may be different than for conventional museums. This research unfolds through a designerly approach to explore the potential of tangible interaction by means of a series of design interventions where art and design practices (e.g. creation of interpretive object), technology (e.g. tangible technologies embedded within object) and historical content (e.g. evocative narrative) are connected together to prompt visitors' personal, tangible and multi-sensory engagement at a house museum. This research is at an early stage (begun in October 2015), thus this paper presents an initial analysis of literature to frame the research, the motivations and context behind the project, the methods to achieve the goals of the study and future work plans.

Punto di arrivo, di Valeria Lomanto e mio, ambizioso, ma per questo affascinante, è corredare il CD nelle future edizioni della possibilità di cercare tutti i passi di un autore ovvero un estremo singolo (ad es. Verg. Aen. 1, 1),... more

Punto di arrivo, di Valeria Lomanto e mio, ambizioso, ma per questo affascinante, è corredare il CD nelle future edizioni della possibilità di cercare tutti i passi di un autore ovvero un estremo singolo (ad es. Verg. Aen. 1, 1), affiancandovi il suddetto prontuario, con tutti i links automatici necessari. L’impresa potrà essere di qualche aiuto sia allo studioso dei Grammatici, in grado di maneggiare uno strumento di ricerca ancor più ricco dell'attuale, sia al filologo ed all'editore di testi in frammenti, che potrà sfruttare il prontuario per meglio impostare sillogi di uno o più autori conosciuti per tradizione indiretta, La divisione dei GL in libri rende inoltre possibile una realizzazione a tappe del progetto, ciascuna coerente in sé e autosufficiente e, soprattutto, la collaborazione di diversi studiosi.

Due to a broad conceptual usage of the term embodiment across a diverse variety of research domains, existing embodied learning games and simulations utilize a large breadth of design approaches that often result in seemingly unrelated... more

Due to a broad conceptual usage of the term embodiment across a diverse variety of research domains, existing embodied learning games and simulations utilize a large breadth of design approaches that often result in seemingly unrelated systems. This becomes problematic when trying to critically evaluate the usage and effectiveness of embodiment within existing designs, as well as when trying to utilize embodiment in the design of new games and simulations. In this paper, we present our work on combining differing conceptual and design approaches for embodied learning systems into a unified design framework. We describe the creation process for the framework, explain its dimensions, and provide examples of its use. Our design framework will benefit educational game researchers by providing a unifying foundation for the description, categorization, and evaluation of designs for embodied learning games and simulations.

Moving from a design perspective, the paper explores the potential of tangible interaction in giving shape to intangible contents in museums and temporary exhibitions. Going beyond tangibility intended in the strict sense of touching... more

Moving from a design perspective, the paper explores the potential of tangible interaction in giving shape to intangible contents in museums and temporary exhibitions.
Going beyond tangibility intended in the strict sense of touching assets we use here a wider interpretation of tangibility that considers touch in the sense of embodied experience. In this way we consider as tangible all those experiences that foster a strong involvement of the body when interacting with digital content. This includes objects-based and gestures- based interactions.
Tangible interaction is interpreted as a practice able to multiply the levels of the narrative, to make the visit experience memorable and to give materiality to intangible values. This approach uses tangible interaction as a way to let the audience experience practices and rituals linked to the contents and representative of the intangible values embedded in the assets.
The potential of tangible interaction to foster the intangible values of cultural heritage is discussed starting from a provisional classification of tangible interaction case studies. In particular four different categories are identified that are: smart replicas/originals, symbolic objects, codified gestures and performing gestures.
In conclusion, two possible design strategies that employ tangible interaction for enabling the experience of intangible values of cultural heritage are highlighted. These are:
Embedding meaning: it consists in creating sensorised objects that embed in themselves meanings related to intangible values of cultural heritage, and that communicate explicitly this meaning in their physicality;
Embodying meaning: it consists in integrating a meaning related to intangible values in gestures, so that intangible values are communicated implicitly in the action performed by the visitor.

This paper examines the under-explored role of video demos as a consequential form of communicating tangible computing research. Examining the production of video demos, in particular two videos from MIT's Media Lab, I explore the ethical... more

This paper examines the under-explored role of video demos as a consequential form of communicating tangible computing research. Examining the production of video demos, in particular two videos from MIT's Media Lab, I explore the ethical complications of producing demo videos by bringing to the surface the tensions inherent in video representations of research, as understood from a social studies of technology perspective. In doing so, this paper hopes to generate discussion about the challenges and responsibilities of creating accurate and compelling narratives around technology research.

This paper presents current research on the design and evaluation of tangible interaction within house museums – a particular type of heritage site. Containers of Stories was an interactive installation co-designed with the volunteers at... more

This paper presents current research on the design and evaluation of tangible interaction within house museums – a particular type of heritage site. Containers of Stories was an interactive installation co-designed with the volunteers at the Bishops’ House museum, one of the few surviving Tudor buildings in Sheffield, UK. Dating from the 16th century, the house was turned into a museum in the 1970s for its historical and social importance and is now managed by a community of local volunteers who constantly needs to increase both visitors’ interest and awareness of the place for its survival. The experience of co-designing Containers of Stories pushed the volunteers beyond day-to-day management toward more creative and curatorial roles. This paper gives insights on how this hybrid physical-digital installation succeeded in engaging visitors in new ways with heritage that has the potential to strengthen the resilience of the community. (Paper to be published in spring 2017)

We present ArtVis, an application combining advanced visualisation techniques and tangible interaction to explore a large digital collection of almost 28000 European artworks managed by the Web Gallery of Art. In order to get new insights... more

We present ArtVis, an application combining advanced visualisation techniques and tangible interaction to explore a large digital collection of almost 28000 European artworks managed by the Web Gallery of Art. In order to get new insights by exploring, analysing and browsing the artworks, our graphical ArtVis user interface offers three complementary but synchronised visualisation components. We further developed a tangible ArtVis user interface for the playful exploration and seamless integration of the digital artwork collection with physical artefacts. A formative evaluation of the ArtVis prototype revealed that users are able to answer relatively difficult questions as well as get some new insights based on the vast amount of data. A second user evaluation of the tangible ArtVis interface has shown that this sort of physical interaction attracts users and stimulates them to further explore the digital artwork collection.

Existing embodied learning games and simulations utilize a large breadth of design approaches that often result in the creation of seemingly unrelated systems. This becomes problematic when trying to critically evaluate the usage and... more

Existing embodied learning games and simulations utilize a large breadth of design approaches that often result in the creation of seemingly unrelated systems. This becomes problematic when trying to critically evaluate the usage and effectiveness of embodiment within embodied learning designs. In this paper, we present our work on combining differing conceptual and design approaches for embodied learning systems into a unified design framework. We describe the creation process for the framework, explain its dimensions, and provide two examples of its use. Our embodied learning games and simulations framework will benefit HCI researchers by providing a unifying foundation for the description, categorization, and evaluation of embodied learning systems and designs.

In this paper we introduce and discuss the nature of free- play in the context of three open-ended interactive art installation works. We observe the interaction work of situated free-play of the participants in these environments and,... more

In this paper we introduce and discuss the nature of free- play in the context of three open-ended interactive art installation works. We observe the interaction work of situated free-play of the participants in these environments and, building on precedent work, devise a set of sensitising terms derived both from the literature and from what we observe from participants interacting there. These sensitising terms act as guides and are designed to be used by those who experience, evaluate or report on open-ended interactive art. That is, we propose these terms as a common-ground language to be used by participants communicating while in the art work to describe their experience, by researchers in the various stages of research process (observation, coding activity, analysis, reporting, and publication), and by inter-disciplinary researchers working across the fields of HCI and art. This work builds a foundation for understanding the relationship between free-play, open-ended environments, and interactive installations and contributes sensitising terms useful for the HCI community for discussion and analysis of open-ended interactive art works.

Meanings and values of built heritage vary from factual and explicit meanings which are relatively easy to present, to more tacit knowledge, which is typically more challenging to communicate due to its implicit and often abstract... more

Meanings and values of built heritage vary from factual and explicit meanings which are relatively easy to present, to more tacit knowledge, which is typically more challenging to communicate due to its implicit and often abstract character. In this paper, we investigate how tangible interaction influences the communication of this tacit knowledge of built heritage, and how it affects the experience of visitors. Through a between-group comparative study in a real-world museum context, we examined how the tangible characteristics of an interactive prototype museum installation influence how visitors perceive a particular story containing tacit heritage knowledge. The communicated story relates a historical journey in ancient Egypt to the physical and architectural characteristics of the entrance colonnade at the Djoser Complex in Saqqara. Our experimental conditions consisted of an interactive navigation (input) and a passive representation (output) components, ranging from traditional digital displays to fully tangible means of interaction. We report on our findings, which showed various differences and commonalities between our three experimental conditions. We conclude with a number of discussion points and design recommendations: (a) to strive for balance between navigation and representation modalities in terms of affordance and the required cognitive effort; (b) to take advantage of physical representation and grasping, such as conveying particular physical details and characteristics; and (c) to consider design aspects of embodiment, physical abstraction and materiality for future research or potential further development of communicating the meanings and values of heritage.

This essay proposes that the electronic texts of plays constituting a database-collection (in this case early modern drama) should be “annotated” by marking up not only its structural components but also the editorial annotations about a... more

This essay proposes that the electronic texts of plays constituting a database-collection (in this case early modern drama) should be “annotated” by marking up not only its structural components but also the editorial annotations about a given feature or aspect of the play (usually included in the commentary notes of print editions), and that these annotations should be conceived having in mind the functionalities of a database. By marking up both the text's structural components and editor's information they constitute related data to be processed by the computer for searches and statistical analysis. This implies that texts should not be annotated individually and independently from the other anthologized works, but rather as part of an organized collection of data that, adequately encoded, will allow users to make queries into the whole database. A second section of the essay discusses three encoding mechanisms, based on the Guidelines of the Text Encoding Initiative, necessary to mark up these “annotations,” and possible ad hoc extensions of the TEI schema in order to represent the annotated features. Finally, a third section comments on practical examples showing how to encode a set of features: scene location, image, theme, allusion, proverb, wordplay, grammar, swearing expression, address form, as well as features covered by the TEI Guidelines such as roles, stage directions, names and place-names, verse form and textual issues.

The Web has changed the way that organisations work around the world and the case of CH is no exception. Many Web technologies such as the Semantic Web and Design protocols have helped managing vast sets of information. On one hand, there... more

The Web has changed the way that organisations work around the world and the case of CH is no exception. Many Web technologies such as the Semantic Web and Design protocols have helped managing vast sets of information. On one hand, there has been an immense progress in the way people (and computers) manage and produce information. Many CH organisations have adopted Linked Data Systems as well as Open Data policies backed up with ontologies to enhance inferences and accuracy of the knowledge depicted. Such methods are becoming more widespread across the CH sector with data models such as CIDOC, Dublin Core and EDM among others, allowing their collections to be interconnected through their metadata. On the other hand, people in CH have used a wide range of technologies to ingest, manage and engage with such knowledge. Interfaces have become the main pathway between the knowledge and the user. Nevertheless, in order to interact with such knowledge, particularly on the Web, users have to navigate through interactive methods widely monopolised by Graphical User Interfaces (GUI). Human Computer Interaction (HCI) research has introduced what can be suggested as novel and more optimal ways of interaction through Tangible User Interfaces (TUI). TUIs offer the opportunity to transform digital content into palpable objects. Therefore, TUIs offer physical manipulation of digital information and the user of sensory-motor skills.
This paper presents a Web based Tangible User Interface to explore Cultural Heritage knowledge. It discusses novel approaches to assist users to conceptualise and deliver complex queries to Linked Data repositories. The paper discusses information engagement challenges currently raised in the Cultural Heritage sector (CH) on the Web. Moreover, it introduces Europeana as a case study where Linked Data repositories are used to offer users the opportunity to explore vast sets of knowledge in museums, libraries, and archives across Europe.

SID, et design- & metodeudviklingsprojekt for interaktive sanserum.
(Design lead: Henrik Svarrer Larsen)

This paper discusses the design process of VR/Urban’s public tangible interface SMSlingshot, a real-time system for urban interventions on Media Façades, which we have exhibited in the last few years around the world. In this case study... more

This paper discusses the design process of VR/Urban’s public tangible interface SMSlingshot, a real-time system for urban interventions on Media Façades, which we have exhibited in the last few years around the world. In this case study we investigate how the design collaboration between technologists and industrial designers contributed to the success of the urban intervention. The design process of this ‘product’ has many DIY aspects, with professional industrial designers and technologists becoming expert amateurs, often dealing with problems that pushed them outside of their professional comfort zone. Don’t be afraid of being an amateur!

It’s time to rethink Interaction Design. In the massively augmented Internet of Things world, our current approaches to IxD, AI and Human Centered Design are too solutionistic and task oriented. My collaborator Betti Marenko and I are... more

It’s time to rethink Interaction Design. In the massively augmented Internet of Things world, our current approaches to IxD, AI and Human Centered Design are too solutionistic and task oriented. My collaborator Betti Marenko and I are proposing Animistic Design as an alternative. We believe that designers can use this new approach to shift focus and create systems that better support creativity, labor, learning, and collaboration.

A prevalent assumption behind interface approaches that employ physical means of interaction is that this leverages users' prior knowledge from the real world. This paper scrutinizes the assumption that this knowledge can be seamlessly... more

A prevalent assumption behind interface approaches that employ physical means of interaction is that this leverages users' prior knowledge from the real world. This paper scrutinizes the assumption that this knowledge can be seamlessly transferred to computer-augmented situations. TEI needs design strategies that acknowledge the hybrid nature of our systems. A change of focus is advocated: from support of intuitive use to the design of seamful mappings and the support of reflection and learning to enable appropriation and a better understanding of the systems we use.

The “One-Person Choir” is a human–computer interface for singers that facilitates gestural control over a digital signal processing (DSP) module for harmonizing the singing voice in real time (see Figure 1). Harmonization adds extra... more

The “One-Person Choir” is a human–computer interface for singers that facilitates gestural control over a digital signal processing (DSP) module for harmonizing the singing voice in real time (see Figure 1). Harmonization adds extra pitch-shifted voices that are tonally related to the input voice. The interface captures global movements of the upper limbs by means of an integrated network of inertial sensors attached to the upper body of a singer. From these data, gestural cues are extracted and compared with a preconfigured gestural model that has been trained with empirical data. When the gestures of the singer match the preconfigured model, it is possible to control the harmonization of the singing input voice captured by a microphone. Thus, the interface allows a singer to naturally enhance the expressive qualities of his or her voice with the assistance of expressive gestures connected to an electronic environment. The One-Person Choir can be integrated in interactive multimedia installations that exploit the expressive power of gestures in combination with singing. As will be argued in this article, installations illustrate, and elaborate on, an ongoing shift in contemporary electronic and electroacoustic music: the move from interactive systems (or hyperinstruments) to composing interactions (Di Scipio 2003).

Touchscreen technologies will most probably replace current instrument panels in future aeronautical cockpits. However, while safety and performance require interactive instruments to maximize the perception, action and collaboration... more

Touchscreen technologies will most probably replace current instrument panels in future aeronautical cockpits. However, while safety and performance require interactive instruments to maximize the perception, action and collaboration spaces offered to pilots, the literature highlights the limits of touch-based interaction regarding these aspects. Our objective is thus to explore how tangible embodied interaction (TEI), associated with a touch-based flight deck device, could address this issue. In this paper, we contribute a structured design space for pilot-system interactions based on an analysis of the design properties of physical interaction as described in the literature, and on relevant usability, safety and industrial requirements.

We introduce an embodied-interaction instructional design, the Mathematical Imagery Trainer (MIT), for helping young students develop grounded understanding of proportional equivalence (e.g., 2/3 = 4/6). Taking advantage of the low-cost... more

We introduce an embodied-interaction instructional design, the Mathematical Imagery Trainer (MIT), for helping young students develop grounded understanding of proportional equivalence (e.g., 2/3 = 4/6). Taking advantage of the low-cost availability of hand-motion tracking provided by the Nintendo Wii remote, the MIT applies cognitive-science findings that mathematical concepts are grounded in mental simulation of dynamic imagery, which is acquired through perceiving, planning, and performing actions with the body. We describe our rationale for and implementation of the MIT through a design-based research approach and report on clinical interviews with twenty-two 4th–6th grade students who engaged in problem-solving tasks with the MIT.

This paper presents the research project The Notion of Participative and Enacting Sonic Interaction (PESI). PESI aims to extend the engagement of performers in collective music practices using embodied approaches in physical and social... more

This paper presents the research project The Notion of Participative and Enacting Sonic Interaction (PESI). PESI aims to extend the engagement of performers in collective music practices using embodied approaches in physical and social interaction. In its design, the mobile phone functions as a tangible and expressive musical instrument, together with an extended system. In this paper, we contextualize the project with associated themes – physical, spatial and social interaction – and with related works. We also present the modular structure of the project, evaluation methods, initial conclusions and paths for future developments.

This paper presents the actual stage of implementation of a new type of malleable tangible object for musical expression. Sculpton is an autonomous sonic object which uses the metaphor of sound sculpting for connecting physical... more

This paper presents the actual stage of implementation of a new type of malleable tangible object for musical expression. Sculpton is an autonomous sonic object which uses the metaphor of sound sculpting for connecting physical information in digital audio. By manipulating the object the user can literally sculpt the sound through a real-time sound synthesis which reverberates the object structure. This project explores a novel approach compared to previous work and research around the topic of sound sculpting: user gestures is not externally sensed but within the artifact itself. Sculpton is an attempt for the development of a new kind of embodied musical instruments which combine multidimensional control, tangible and malleable characteristics with an organic handling. We will briefly describe the research, our framework and the actual state of the experimental prototypes.

Today, it is not uncommon to find ourselves remote from those we care about. Despite the impact of mobile and social technologies on connectedness, recent studies suggest that it could be these very technologies that exacerbate a sense of... more

Today, it is not uncommon to find ourselves remote from those we care about. Despite the impact of mobile and social technologies on connectedness, recent studies suggest that it could be these very technologies that exacerbate a sense of loneliness. In attempt to help people feel more connected, we designed and created BodyPods, a remotely paired set of communicating chairs that facilitate a sense of presence by leveraging implicit actions such as sitting to communicate that someone you care about is home. Each BodyPod consists of a flexible surface with six pressure-sensitive and light-emitting pads that adjusts its shape to the body anatomy. As a person’s body moves, limbs exert different pressure on each pad creating a live digital “bodyprint” that is mapped on the pads of other BodyPods through color and light. Findings from a 10 person user study suggest bodyprints may be distinctive, particularly among small groups of people with different body types.

This paper presents the prototype of a reactive tangible surface we call Sensitive Surface. It is an interactive surface system, built out of individually controllable actuator cells, which can produce output for different aspects of our... more

This paper presents the prototype of a reactive tangible surface we call Sensitive Surface. It is an interactive surface system, built out of individually controllable actuator cells, which can produce output for different aspects of our sense of touch. It enables a variety of actuation patterns by overlaying different actuator combinations and intensities. We explain the concept and motivation behind its design and discuss possible applications. Originally developed for use in interactive architecture scenarios, Sensitive Surface also has potential applications in therapeutic and medical environments. One main line of investigation we explored with the prototype was the notion of an individual touch aesthetic, which could be controlled or created through different types of user feedback.

Figure 1. GazeForm concept. Left: when eye-hand coordination exists for touch modality the surface is flat. Right: when the gaze is solicited by another task the surface offers a salient tangible control allowing continuity of... more

Figure 1. GazeForm concept. Left: when eye-hand coordination exists for touch modality the surface is flat. Right: when the gaze is solicited by another task the surface offers a salient tangible control allowing continuity of manipulation and freeing the gaze. ABSTRACT An increasing number of domains, including aeronautics, are adopting touchscreens. However, several drawbacks limit their operational use, in particular, eyes-free interaction is almost impossible making it difficult to perform other tasks simultaneously. We introduce GazeForm, an adaptive touch interface with shape-changing capacity that offers an adapted interaction modality according to gaze direction. When the user's eyes are focused on interaction, the surface is flat and the system acts as a touchscreen. When eyes are directed towards another area, physical knobs emerge from the surface. Compared to a touch only mode, experimental results showed that GazeForm generated a lower subjective mental workload and a higher efficiency of execution (20% faster). Furthermore, GazeForm required less visual attention and participants were able to concentrate more on a secondary monitoring task. Complementary interviews with pilots led us to explore timings and levels of control for using gaze to adapt modality.

There is a strong relation between playful learning and the environment children inhabit. An environment can easily turn into a medium for play while its patterns and children's interactions with these patterns can turn into experiences... more

There is a strong relation between playful learning and the environment children inhabit. An environment can easily turn into a medium for play while its patterns and children's interactions with these patterns can turn into experiences of learning. Developed upon findings from an analog pilot study and built with an open source electronic platform and piezoelectric sensors, our prototype translates children's physical actions into responses of an interactive device. Experiments with children using the prototype support, in part, a unifying approach to designing playful learning environments embedded with tangible spatial interaction.

This paper explores the potential of tangible and embodied interaction for encouraging a multisensory engagement with museum objects and artefacts on display, by means of focusing on the subtleties of devising and planning for evaluation... more

This paper explores the potential of tangible and embodied interaction for encouraging a multisensory engagement with museum objects and artefacts on display, by means of focusing on the subtleties of devising and planning for evaluation and audience research. Measuring the impact of new technologies is one of the main challenges identified in the 2015 NMC Horizon report (Museum Edition). The challenge is even greater for emerging concepts, technologies, and approaches, such as the use of tangible and embodied interaction in museums and other Cultural Heritage settings. Taking as an example two case-studies from the EU meSch project, from Museon and Allard Pierson Museum in the Netherlands, we discuss our plan for devising and carrying out audience research so as to " document, " analyse, and interpret the impact of digitally enhanced, tangible, embodied, and multisensory museum visiting experiences. Our intention is to provide an honest account of the different strengths and weaknesses encountered for all evaluation methodologies that were used, namely observations, interviews, video data, questionnaires, meaning maps, and post-visit interviews. We also share and discuss lessons learned, insights and best practices that could be of benefit for museum and audience research professionals.