Zooids | Proceedings of the 29th Annual Symposium on User Interface Software and Technology (original) (raw)

Zooids: Building Blocks for Swarm User Interfaces

Published: 16 October 2016 Publication History

Abstract

This paper introduces swarm user interfaces, a new class of human-computer interfaces comprised of many autonomous robots that handle both display and interaction. We describe the design of Zooids, an open-source open-hardware platform for developing tabletop swarm interfaces. The platform consists of a collection of custom-designed wheeled micro robots each 2.6 cm in diameter, a radio base-station, a high-speed DLP structured light projector for optical tracking, and a software framework for application development and control. We illustrate the potential of tabletop swarm user interfaces through a set of application scenarios developed with Zooids, and discuss general design considerations unique to swarm user interfaces.

Supplementary Material

suppl.mov (uist2890-file3.mp4)

• Download
• 47.42 MB

MP4 File (p97-le-goc.mp4)

• Download
• 148.08 MB

References

[1]

Ahlberg, C., Williamson, C., and Shneiderman, B. Dynamic queries for information exploration: An implementation and evaluation. In Proceedings of the SIGCHI conference on Human factors in computing systems, ACM (1992), 619--626.

[2]

Alonso-Mora, J., Breitenmoser, A., Rufli, M., Siegwart, R., and Beardsley, P. Multi-robot system for artistic pattern formation. In Robotics and Automation (ICRA), 2011 IEEE International Conference on, IEEE (2011), 4512--4517.

[3]

Alonso-Mora, J., Lohaus, S. H., Leemann, P., Siegwart, R., and Beardsley, P. Gesture based human-multi-robot swarm interaction and its application to an interactive display. In 2015 IEEE International Conference on Robotics and Automation (ICRA), IEEE (2015), 5948--5953.

[4]

Amano, K., and Yamamoto, A. Tangible interactions on a flat panel display using actuated paper sheets. In Proceedings of the 2012 ACM international conference on Interactive tabletops and surfaces, ACM (2012), 351--354.

[5]

Bennett, E., and Stevens, B. The effect that touching a projection augmented model has on object-presence. In Information Visualisation, 2005. Proceedings. Ninth International Conference on, IEEE (2005), 790--795.

[6]

Brave, S., Ishii, H., and Dahley, A. Tangible interfaces for remote collaboration and communication. In Proceedings of the 1998 ACM conference on Computer supported cooperative work, ACM (1998), 169--178.

[7]

Chang, B.-W., and Ungar, D. Animation: from cartoons to the user interface.

[8]

Cucu, L., Rubenstein, M., and Nagpal, R. Towards self-assembled structures with mobile climbing robots. In Robotics and Automation (ICRA), 2015 IEEE International Conference on, IEEE (2015), 1955--1961.

[9]

Ducatelle, F., Di Caro, G., Pinciroli, C., and Gambardella, L. Self-organized cooperation between robotic swarms. Swarm Intelligence Journal 5, 2 (2011), 73--96.

[10]

Dudek, G., Jenkin, M., Milios, E., and Wilkes, D. A taxonomy for swarm robots. In Intelligent Robots and Systems'93, IROS'93. Proceedings of the 1993 IEEE/RSJ International Conference on, vol. 1, IEEE (1993), 441--447.

[11]

Felton, S. M., Tolley, M. T., Onal, C. D., Rus, D., and Wood, R. J. Robot self-assembly by folding: A printed inchworm robot. In Robotics and Automation (ICRA), 2013 IEEE International Conference on, IEEE (2013), 277--282.

[12]

Fishkin, K. P. A taxonomy for and analysis of tangible interfaces. Personal Ubiquitous Comput. 8 (September 2004), 347--358.

[13]

Fitzmaurice, G. W., and Buxton, W. An empirical evaluation of graspable user interfaces: towards specialized, space-multiplexed input. In Proc. CHI 1997, 43--50.

[14]

Follmer, S., Leithinger, D., Olwal, A., Cheng, N., and Ishii, H. Jamming User Interfaces: Programmable Particle Stiffness and Sensing for Malleable and Shape-Changing Devices. In ACM Symposium on User Interface Software and Technology (2012), 519--528.

[15]

Follmer, S., Leithinger, D., Olwal, A., Hogge, A., and Ishii, H. inform: Dynamic physical affordances and constraints through shape and object actuation. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, UIST '13, ACM (New York, NY, USA, 2013), 417--426.

[16]

Futurelab, A. E. Drone 100 -- the world record for intel 2015. http://tinyurl.com/drone100, 2016.

[17]

Gervais, R. Interaction and introspection with tangible augmented objects. Phd dissertation, Université de Bordeaux, Dec. 2015.

[18]

Goldstein, S. C., Campbell, J. D., and Mowry, T. C. Programmable matter. Computer 38, 6 (2005), 99--101.

[19]

Greenberg, S., and Fitchett, C. Phidgets: easy development of physical interfaces through physical widgets. In Proceedings of the 14th annual ACM symposium on User interface software and technology, ACM (2001), 209--218.

[20]

Grieder, R., Alonso-Mora, J., Bloechlinger, C., Siegwart, R., and Beardsley, P. Multi-robot control and interaction with a hand-held tablet. In Workshop Proc. Int. Conf. Robotics and Automation, vol. 131, Citeseer (2014).

[21]

Hauri, S., Alonso-Mora, J., Breitenmoser, A., Siegwart, R., and Beardsley, P. Multi-robot formation control via a real-time drawing interface. In Field and Service Robotics, Springer (2014), 175--189.

[22]

Heer, J., and Robertson, G. G. Animated transitions in statistical data graphics. Visualization and Computer Graphics, IEEE Transactions on 13, 6 (2007), 1240--1247.

[23]

Horn, M. S., Solovey, E. T., Crouser, R. J., and Jacob, R. J. Comparing the use of tangible and graphical programming languages for informal science education. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '09, 975--984.

[24]

Hornecker, E., and Buur, J. Getting a grip on tangible interaction: a framework on physical space and social interaction. In Proceedings of the SIGCHI conference on Human Factors in computing systems, ACM (2006), 437--446.

[25]

Huron, S., Jansen, Y., and Carpendale, S. Constructing visual representations: Investigating the use of tangible tokens. Visualization and Computer Graphics, IEEE Transactions on 20, 12 (2014), 2102--2111.

[26]

Ishii, H., Lakatos, D., Bonanni, L., and Labrune, J.-B. Radical atoms: beyond tangible bits, toward transformable materials. interactions 19, 1 (Jan. 2012), 38--51.

[27]

Jansen, Y., Dragicevic, P., and Fekete, J.-D. Evaluating the efficiency of physical visualizations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2013), 2593--2602.

[28]

Jansen, Y., Dragicevic, P., Isenberg, P., Alexander, J., Karnik, A., Kildal, J., Subramanian, S., and Hornbæk, K. Opportunities and challenges for data physicalization. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM (2015), 3227--3236.

[29]

Kira, Z., and Potter, M. A. Exerting human control over decentralized robot swarms. In Autonomous Robots and Agents, 2009. ICARA 2009. 4th International Conference on, IEEE (2009), 566--571.

[30]

Kojima, M., Sugimoto, M., Nakaruma, A., Tomita, M., Inami, M., and Nii, H. Augmented coliseum: An augmented game environment with small vehicles. Horizontal Interactive Human-Computer Systems, International Workshop on 0 (2006), 3--8.

[31]

Kolling, A., Nunnally, S., and Lewis, M. Towards human control of robot swarms. In Proceedings of the seventh annual ACM/IEEE international conference on human-robot interaction, ACM (2012), 89--96.

[32]

Kuhn, H. W. The hungarian method for the assignment problem. Naval research logistics quarterly 2, 1--2 (1955), 83--97.

[33]

Kushleyev, A., Mellinger, D., Powers, C., and Kumar, V. Towards a swarm of agile micro quadrotors. Autonomous Robots 35, 4 (2013), 287--300.

[34]

Le Goc, M., Dragicevic, P., Huron, S., Boy, J., and Fekete, J.-D. Smarttokens: Embedding motion and grip sensing in small tangible objects. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, ACM (2015), 357--362.

[35]

Le Goc, M., Dragicevic, P., Huron, S., Boy, J., and Fekete, J.-D. A better grasp on pictures under glass: Comparing touch and tangible object manipulation using physical proxies. In Proceedings of the International Working Conference on Advanced Visual Interfaces, ACM (2016), 76--83.

[36]

Lederman, S. J., and Campbell, J. I. Tangible graphs for the blind. Human Factors: The Journal of the Human Factors and Ergonomics Society 24, 1 (1982), 85--100.

[37]

Lee, J. C., Hudson, S. E., Summet, J. W., and Dietz, P. H. Moveable interactive projected displays using projector based tracking. In Proceedings of the 18th annual ACM symposium on User interface software and technology, ACM (2005), 63--72.

[38]

Lee, N., Kim, J., Lee, J., Shin, M., and Lee, W. Molebot: mole in a table. In ACM SIGGRAPH 2011 Emerging Technologies, ACM (2011), 9.

[39]

Leithinger, D., and Ishii, H. Relief: a scalable actuated shape display. In Proceedings of the fourth international conference on Tangible, embedded, and embodied interaction, ACM (2010), 221--222.

[40]

Lifton, J., Broxton, M., and Paradiso, J. A. Experiences and directions in pushpin computing. In IPSN 2005. Fourth International Symposium on Information Processing in Sensor Networks, 2005., IEEE (2005), 416--421.

[41]

Marquardt, N., Nacenta, M. A., Young, J. E., Carpendale, S., Greenberg, S., and Sharlin, E. The haptic tabletop puck: tactile feedback for interactive tabletops. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, ACM (2009), 85--92.

[42]

Marshall, M., Carter, T., Alexander, J., and Subramanian, S. Ultra-tangibles: creating movable tangible objects on interactive tables. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2012), 2185--2188.

[43]

Mi, H., and Sugimoto, M. Hats: interact using height-adjustable tangibles in tabletop interfaces. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, ACM (2011), 71--74.

[44]

Microsoft. Sanddance: Visually explore, understand, and present data. Online. http://research.microsoft.com/en-us/projects/sanddance/, 2016.

[45]

Moere, A. V. Beyond the tyranny of the pixel: Exploring the physicality of information visualization. In Information Visualisation, 2008. IV'08. 12th International Conference, IEEE (2008), 469--474.

[46]

Nielsen, J. Usability engineering.

[47]

Nowacka, D., Ladha, K., Hammerla, N. Y., Jackson, D., Ladha, C., Rukzio, E., and Olivier, P. Touchbugs: Actuated tangibles on multi-touch tables. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2013), 759--762.

[48]

Pangaro, G., Maynes-Aminzade, D., and Ishii, H. The actuated workbench: Computer-controlled actuation in tabletop tangible interfaces. In Proceedings of the 15th Annual ACM Symposium on User Interface Software and Technology, UIST '02, 181--190.

[49]

Patten, J. Thumbles - robotic tabletop user interface platform. TED.com (2014).

[50]

Patten, J., and Ishii, H. Mechanical constraints as computational constraints in tabletop tangible interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '07, ACM (New York, NY, USA, 2007), 809--818.

[51]

Patten, J., Ishii, H., Hines, J., and Pangaro, G. Sensetable: A wireless object tracking platform for tangible user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '01, ACM (New York, NY, USA, 2001), 253--260.

[52]

Patten, J., Recht, B., and Ishii, H. Audiopad: A tag-based interface for musical performance. In Proceedings of the 2002 Conference on New Interfaces for Musical Expression, NIME '02, National University of Singapore (Singapore, Singapore, 2002), 1--6.

[53]

Pedersen, E. W., and Hornbæk, K. Tangible bots: interaction with active tangibles in tabletop interfaces. In Proc. CHI, ACM (2011), 2975--2984.

[54]

Poupyrev, I., Nashida, T., Maruyama, S., Rekimoto, J., and Yamaji, Y. Lumen: Interactive visual and shape display for calm computing. In ACM SIGGRAPH 2004 Emerging Technologies, SIGGRAPH '04, ACM (New York, NY, USA, 2004), 17--.

[55]

Poupyrev, I., Nashida, T., and Okabe, M. Actuation and tangible user interfaces: the vaucanson duck, robots, and shape displays. In TEI '07, 205--212.

[56]

Rasmussen, M. K., Pedersen, E. W., Petersen, M. G., and Hornbaek, K. Shape-changing interfaces: a review of the design space and open research questions. In CHI '12, 735--744.

[57]

Reznik, D., and Canny, J. A flat rigid plate is a universal planar manipulator. In IEEE ICRA 1998, vol. 2, IEEE (1998), 1471--1477.

[58]

Richter, J., Thomas, B. H., Sugimoto, M., and Inami, M. Remote active tangible interactions. In Proceedings of the 1st international conference on Tangible and embedded interaction, ACM (2007), 39--42.

[59]

Romanishin, J. W., Gilpin, K., and Rus, D. M-blocks: Momentum-driven, magnetic modular robots. In Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on, IEEE (2013), 4288--4295.

[60]

Rosenfeld, D., Zawadzki, M., Sudol, J., and Perlin, K. Physical objects as bidirectional user interface elements. Computer Graphics and Applications, IEEE 24, 1 (2004), 44--49.

[61]

Roudaut, A., Karnik, A., Löchtefeld, M., and Subramanian, S. Morphees: toward high shape resolution in self-actuated flexible mobile devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2013), 593--602.

[62]

Rubens, C., Braley, S., Gomes, A., Goc, D., Zhang, X., Carrascal, J. P., and Vertegaal, R. Bitdrones: Towards levitating programmable matter using interactive 3d quadcopter displays. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, ACM (2015), 57--58.

[63]

Rubenstein, M., Ahler, C., and Nagpal, R. Kilobot: A low cost scalable robot system for collective behaviors. In Robotics and Automation (ICRA), 2012 IEEE International Conference on, IEEE (2012), 3293--3298.

[64]

Rus, D. Programmable matter with self-reconfiguring robots. In Proceedings of the 7th ACM international conference on Computing frontiers, CF '10, 51--52.

[65]

Sahai, R., Avadhanula, S., Groff, R., Steltz, E., Wood, R., and Fearing, R. S. Towards a 3g crawling robot through the integration of microrobot technologies. In Robotics and Automation, 2006. ICRA 2006. Proceedings 2006 IEEE International Conference on, IEEE (2006), 296--302.

[66]

Seah, S. A., Drinkwater, B. W., Carter, T., Malkin, R., and Subramanian, S. Dexterous ultrasonic levitation of millimeter-sized objects in air. IEEE transactions on ultrasonics, ferroelectrics, and frequency control 61, 7 (2014), 1233--1236.

[67]

Snape, J., van den Berg, J., Guy, S. J., and Manocha, D. The hybrid reciprocal velocity obstacle. Robotics, IEEE Transactions on 27, 4 (2011), 696--706.

[68]

Snape, J., van den Berg, J. P., Guy, S. J., and Manocha, D. Independent navigation of multiple mobile robots with hybrid reciprocal velocity obstacles. In IROS (2009), 5917--5922.

[69]

Steltz, E., Seeman, M., Avadhanula, S., and Fearing, R. S. Power electronics design choice for piezoelectric microrobots. In Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on, IEEE (2006), 1322--1328.

[70]

Sutherland, I. E. The ultimate display, 1965.

[71]

Taher, F., Hardy, J., Karnik, A., Weichel, C., Jansen, Y., Hornbæk, K., and Alexander, J. Exploring interactions with physically dynamic bar charts. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM (2015), 3237--3246.

[72]

Ullmer, B., and Ishii, H. The metadesk: Models and prototypes for tangible user interfaces. In Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology, UIST '97, ACM (New York, NY, USA, 1997), 223--232.

[73]

Ullmer, B., Ishii, H., and Jacob, R. J. K. Token+constraint systems for tangible interaction with digital information. ACM Trans. Comput.-Hum. Interact. 12, 1 (Mar. 2005), 81--118.

[74]

Underkoffier, J., and Ishii, H. Urp: A luminous-tangible workbench for urban planning and design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '99, 386--393.

[75]

Victor, B. A brief rant on the future of interaction design. http://tinyurl.com/bvrant, 2011.

[76]

Wakita, A., Nakano, A., and Kobayashi, N. Programmable blobs: a rheologic interface for organic shape design. In Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction, ACM (2011), 273--276.

[77]

Weiser, M. Some computer science issues in ubiquitous computing. Communications of the ACM 36, 7 (1993), 75--84.

[78]

Weiss, M., Schwarz, F., Jakubowski, S., and Borchers, J. Madgets: Actuating widgets on interactive tabletops. In Proceedings of the 23Nd Annual ACM Symposium on User Interface Software and Technology, UIST '10, 293--302.

[79]

Werfel, J., Petersen, K., and Nagpal, R. Designing collective behavior in a termite-inspired robot construction team. Science 343, 6172 (2014), 754--758.

[80]

Yamamoto, A., Tsuruta, S., and Higuchi, T. Planar 3-dof paper sheet manipulation using electrostatic induction. In Industrial Electronics (ISIE), 2010 IEEE International Symposium on, IEEE (2010), 493--498.

[81]

Yamanaka, S., and Miyashita, H. Vibkinesis: notification by direct tap and-dying message-using vibronic movement controllable smartphones. In Proceedings of the 27th annual ACM symposium on User interface software and technology, ACM (2014), 535--540.

[82]

Yao, L., Niiyama, R., Ou, J., Follmer, S., Della Silva, C., and Ishii, H. Pneui: Pneumatically actuated soft composite materials for shape changing interfaces. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, UIST '13, 13--22.

[83]

Yi, J. S., Melton, R., Stasko, J., and Jacko, J. A. Dust & magnet: multivariate information visualization using a magnet metaphor. Information Visualization 4, 4 (2005), 239--256.

[84]

Zhao, J., and Moere, A. V. Embodiment in data sculpture: a model of the physical visualization of information. In Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts, ACM (2008), 343--350.

Information & Contributors

Information

Published In

UIST '16: Proceedings of the 29th Annual Symposium on User Interface Software and Technology

October 2016

908 pages

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 October 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

1. swarm user interfaces
2. tangible user interfaces

Qualifiers

• Research-article

Funding Sources

• Région Ile de France

Conference

Acceptance Rates

UIST '16 Paper Acceptance Rate 79 of 384 submissions, 21%;

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

• View Citations
• Downloads (Last 12 months)424
• Downloads (Last 6 weeks)35

Reflects downloads up to 31 Dec 2024

Other Metrics

Citations

• Kedwan F(2024)Intermolecular Forces in a Physical Molecular Visualization System: A Human-Computer Interaction ApplicationEuropean Journal of Applied Science, Engineering and Technology10.59324/ejaset.2024.2(2).142:2(212-223)Online publication date: 1-Mar-2024
• Kedwan F(2024)Intermolecular Forces in a Physical Molecular Visualization System: A Human-Computer Interaction ApplicationSSRN Electronic Journal10.2139/ssrn.4840189Online publication date: 2024
• Pulatova SKim L(2024)Co-Designing Programmable Fidgeting Experience with Swarm Robots for Adults with ADHDProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675614(1-14)Online publication date: 27-Oct-2024
• Retnanto AFaracci ESathya AHung YNakagaki K(2024)CARDinality: Interactive Card-shaped Robots with Locomotion and Haptics using VibrationProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676421(1-14)Online publication date: 13-Oct-2024
• Wang KWang ZNakagaki KPerlin K(2024)“Push-That-There”: Tabletop Multi-robot Object Manipulation via Multimodal 'Object-level Instruction'Proceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661542(2497-2513)Online publication date: 1-Jul-2024
• Bakhtiari BPerin CSomanath S(2024)VISMOCK: A Programmable Smocking Technique for Creating Interactive Data PhysicalizationProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660749(2341-2356)Online publication date: 1-Jul-2024
• Bhattacharya RLi YFaracci EDong HZheng YNakagaki K(2024)Threading Space: Kinetic Sculpture Exploring Spatial Interaction Using Threads In MotionProceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3660498(571-575)Online publication date: 23-Jun-2024
• Hunt CSun KTseng KBalasubramaniyam PDruin AHuynh ALeithinger DYip J(2024)Making a Metaphor Sandwich: Analyzing Children's use of Metaphor During Tabletop Telepresence Robot Supported Participatory DesignProceedings of the 23rd Annual ACM Interaction Design and Children Conference10.1145/3628516.3656272(173-188)Online publication date: 17-Jun-2024
• Goto TKadomoto JIrie HSakai SCiolfi LHogan TDöring TJenkins Tvan Dijk JHuron SLi ZCoyle DSigner B(2024)Shape Estimation Algorithm for Collective Shape-Changing Interface Using Wirelessly Connected ComputersProceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3623509.3635267(1-7)Online publication date: 11-Feb-2024
• Bhattacharya RLindstrom JTaka ANisser MMueller SNakagaki KCiolfi LHogan TDöring TJenkins Tvan Dijk JHuron SLi ZCoyle DSigner B(2024)FabRobotics: Fusing 3D Printing with Mobile Robots to Advance Fabrication, Robotics, and InteractionProceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3623509.3633365(1-13)Online publication date: 11-Feb-2024
• Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Sign in

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Affiliations

Mathieu Le Goc

Inria, Université Paris-Sud, & Université Paris-Saclay, Saclay, France

Lawrence H. Kim

Stanford University, Stanford, CA, USA

Ali Parsaei

Stanford University, Stanford, CA, USA

Jean-Daniel Fekete

Inria & Université Paris-Saclay, Saclay, France

Pierre Dragicevic

Inria & Université Paris-Saclay, Saclay, France

Sean Follmer

Stanford University, Stanford, CA, USA