FaceMouse: A Human-Computer Interface for Tetraplegic People (original) (raw)

Abstract

This paper proposes a new human-machine interface particularly conceived for people with severe disabilities (specifically tetraplegic people), that allows them to interact with the computer for their everyday life by means of mouse pointer. In this system, called FaceMouse, instead of classical "pointer paradigm" that requires the user to look at the point where to move, we propose to use a paradigm called "derivative paradigm", where the user does not indicate the precise position, but the direction along which the mouse pointer must be moved. The proposed system is composed of a common, low-cost webcam, and by a set of computer vision techniques developed to identify the parts of the user’s face (the only body part that a tetraplegic person can move) and exploit them for moving the pointer. Specifically, the implemented algorithm is based on template matching to track the nose of the user and on cross-correlation to calculate the best match. Finally, several real applications of the system are described and experimental results carried out by disabled people are reported.

Chapter PDF

Similar content being viewed by others

References

  1. Beach, G., Cohen, C.J., Braun, J., Moody, G.: Eye tracker system for use with head mounted displays. In: Proceedings of IEEE Intl Conf on Systems, Man, and Cybernetics, vol. 5, pp. 4348–4352 (1998)
    Google Scholar
  2. Zhu, Z., Ji, Q., Fujimura, K., Lee, K.: Combining Kalman filtering and mean shift for real time eye tracking under active IR illumination. In: Proceedings of Intl Conf on Pattern Recognition, vol. 4, pp. 318–321 (2002)
    Google Scholar
  3. Tu, J., Huang, T., Tao, H.: Face as mouse through visual face tracking. In: Proceedings of 2nd Canadian Conf on Computer and Robot Vision, pp. 339–346 (2005)
    Google Scholar
  4. Qian, R.J., Sezan, M.I., Matthews, K.E.: A Robust Real-Time Face Tracking Algorithm. In: Proceedings of IEEE Intl Conf on Image Processing, vol. 1, pp. 131–135 (1998)
    Google Scholar
  5. Toyama, K.: Look, ma – no hands! Hands free cursor control with real-time 3D face tracking. In: Proceedings of Workshop on Perceptual User Interfaces, pp. 49–54 (1998)
    Google Scholar
  6. Hutchinson, T.E., White, K.P., Martin, W.N., Reichert, K.C., Frey, L.A.: Human-computer interaction using eye-gaze input. IEEE Transactions on Systems, Man, and Cybernetics 19, 1527–1534 (1989)
    Article Google Scholar
  7. Betke, M., Gips, J., Fleming, P.: The Camera Mouse: Visual Tracking of Body Features to Provide Computer Access for People With Severe Disabilities. IEEE Transactions on Neural Systems and Rehabilitation Engineering 10(1), 1–10 (2002)
    Article Google Scholar
  8. Gorodnichy, D.O., Malik, S., Roth, G.: Use Your Nose as a Mouse – a New Technology for Hands-free Games and Interfaces. Computational Video Group, IIT, National Research Council, Ottawa, Canada K1A 0R6
    Google Scholar
  9. Verma, R., Schmid, C., Mikolajczyk, K.: Face detection and tracking in a video by Propagatine Detection Probabilities. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(10), 1215–1228 (2003)
    Article Google Scholar
  10. Moghaddam, B., Pentland, A.: Probabilistic Visual Learning for Object Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7), 696–710 (1997)
    Article Google Scholar
  11. Kalman, R.E.: A new approach to linear filtring and prediction problems. Journal of Basic Engineering, Trans. of ASME (1960)
    Google Scholar
  12. Oliver, N., Pentland, A.P., Berard, F.: LAFTER: lips and face real time tracker. In: Proceedings of Computer Vision and Pattern Recognition, pp. 123–129 (1997)
    Google Scholar
  13. Fagiani, C., Betke, M., Gips, J.: Evaluation of Tracking Methods for Human-Computer Interaction. In: Proceedings of Workshop on Applications of Computer Vision (2002)
    Google Scholar
  14. Campedelli, P., Casiraghi, E., Lanzerotti, R.: Detection of Facial Features. Technical Report DSI, University of Milan
    Google Scholar
  15. Gorodnichy, D.: On importance of nose for face tracking. In: Proceedings of Intl Conf on Automatic Face and Gesture Recognition (2002)
    Google Scholar
  16. Horn, B.K.P.: Robot Vision. MIT Press, Cambridge (1986)
    Google Scholar
  17. Betke, M., Haritaoglu, E., Davis, L.S.: Real-time multiple vehicle detection and tracking from a moving vehicle. Machine Vision and Applications 12(2), 69–83 (2000)
    Article Google Scholar

Download references

Author information

Authors and Affiliations

  1. Dipartimento di Ingegneria dell’Informazione, University of Modena and Reggio Emilia, Italy Via Vignolese 905, Modena, Italy
    Emanuele Perini, Simone Soria, Andrea Prati & Rita Cucchiara

Authors

  1. Emanuele Perini
  2. Simone Soria
  3. Andrea Prati
  4. Rita Cucchiara

Editor information

Editors and Affiliations

  1. Beckman Institute, University of Illinois at Urbana-Champaign, USA
    Thomas S. Huang
  2. Intelligent Systems Lab Amsterdam, University of Amsterdam, The Netherlands
    Nicu Sebe
  3. LIACS Media Lab, Leiden University, Netherlands
    Michael S. Lew
  4. Deptartment of Computer Science, Rutgers University, 08854, Piscataway, NJ, USA
    Vladimir Pavlović
  5. Naval Postgraduate School, USA
    Mathias Kölsch
  6. School of Computing, University of Leeds, LS2 9JT, UK
    Aphrodite Galata
  7. Delphi Coorporation, USA
    Branislav Kisačanin

Rights and permissions

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Perini, E., Soria, S., Prati, A., Cucchiara, R. (2006). FaceMouse: A Human-Computer Interface for Tetraplegic People. In: Huang, T.S., et al. Computer Vision in Human-Computer Interaction. ECCV 2006. Lecture Notes in Computer Science, vol 3979. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11754336\_10

Download citation

Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Publish with us