A Virtual Reality-Based Framework for Experiments on Perception of Manual Gestures (original) (raw)

Abstract

This work contributes an integrated and flexible approach to sign language processing in virtual environments that allows for interactive experimental evaluations with high ecological validity. Initial steps deal with real-time tracking and processing of manual gestures. Motion data is stereoscopically rendered in immersive virtual environments with varying spatial and representational configurations. Besides flexibility, the most important aspect is the seamless integration within a VR-based neuropsychological experiment software. Ongoing studies facilitated with this system contribute to the understanding of the cognition of sign language. The system is beneficial for experimenters because of the controlled and immersive three-dimensional environment enabling experiments with visual depth perception that can not be achieved with video presentations.

Loading...

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

References (10)

  1. Fehrmann, G., Jäger, L.: Sprachbewegung und Raumerinnerung. Zur topographis- chen Medialität der Gebärdensprachen. In: Kunst der Bewegung. Kinästhetische Wahrnehmung und Probehandeln in virtuellen Welten. Volume 8 of Publikationen zur Zeitschrift für Germanistik. Peter Lang, Bern (2004) 311-341
  2. Lee, J., Kunii, T.L.: Visual translation: from native language to sign language. In: Proc. of IEEE Workshop on Visual Languages, USA (1992) 103-109
  3. Lebourque, T., Gibet, S.: High level specification and control of communication gestures: the GESSYCA system. In: Proc. of IEEE Computer Animation, Switzer- land (1999) 24-35
  4. Heloir, A., Gibet, S., Multon, F., Courty, N.: Captured Motion Data Processing for Real Time Synthesis of Sign Language. In: Gesture in Human-Computer In- teraction and Simulation, 6th International Gesture Workshop, GW 2005, Revised Selected Papers. (2006) 168-171
  5. Cox, S., Lincoln, M., Tryggvason, J., Nakisa, M., Wells, M., Tutt, M., Abbott, S.: TESSA, a system to aid communication with deaf people. In: Proc. of ASSETS 2002, Fifth International ACM SIGCAPH Conference on Assistive Technologies, Scotland (2002) 205-212
  6. Kennaway, R.: Synthetic Animation of Deaf Signing Gestures. In: GW '01: Revised Papers from the International Gesture Workshop on Gesture and Sign Languages in Human-Computer Interaction. (2002) 146-157
  7. Rezzoug, N., Gorce, P., Hloir, A., Gibet, S., Courty, N., Kamp, J.F., Multon, F., Pelachaud, C.: Virtual humanoids endowed with expressive communication gestures : The HuGEx project. In: Proc. of IEEE International Conference on Systems, Man, and Cybernetics, Taiwan (2006)
  8. Ullrich, S., Valvoda, J.T., Prescher, A., Kuhlen, T.: Comprehensive Architecture for Simulation of the Human Body based on Functional Anatomy. In: Proc. of BVM 2007, Germany (2007)
  9. Valvoda, J.T., Kuhlen, T., Bischof, C.: Interactive Virtual Humanoids for Vir- tual Environments. In: Short Paper Proc. of Eurographics Symposium on Virtual Environments, Portugal (2006) 9-12
  10. Valvoda, J.T., Kuhlen, T., Wolter, M., et al.: NeuroMan: A Comprehensive Soft- ware System for Neuropsychological Experiments. CyberPsychology and Behaviour 8(4) (2005) 366-367