Real-time extraction of surface patches with associated uncertainties by means of Kinect cameras (original) (raw)

Abstract

In this paper, we present our work on GPUbased real-time extraction of surface patches by means of Kinect cameras. This paper makes four contributions: (1) we derive an uncertainty model for pixel-wise depth reconstruction on Kinect cameras; (2) we implement a realtime algorithm for surface patch (here called 'texlet') extraction based on Kinect depth data on a GPU. For that we compare and evaluate different implementation alternatives. (3) Based on (1) we derive and implement an appropriate uncertainty model for texlets which is also computed in real-time. (4) We investigate and quantify the effect of interferences on the depth extraction process when using multiple Kinect cameras. By these contributions we present insights into the processing of depth and how to achieve higher precision reconstructions by means of Kinect cameras as well as extend their use for higher level visual processing. The introduced algorithms are available in the C?? vision library CoViS.

Loading...

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

References (31)

  1. Bas ¸eski, E., Pugeault, N., Kalkan, S., Bodenhagen, L., Piater, J.H., Kru ¨ger, N.: Using multi-modal 3D contours and their rela- tions for vision and robotics. J Vis Commun Image Represent 21(8), 850-864 (2010)
  2. Berger, K., Ruhl, K., Bru ¨mmer C., Schro ¨der, Y., Scholz, A., Magnor, M.: Markerless motion capture using multiple color- depth sensors. In: Proceedings of Vision, Modeling and Visual- ization (VMV) 2011, pp. 317-324, October 2011
  3. Carceroni, R.L., Kutalakos, K.N.: Multi-view scene capture by surfel sampling: From video streams to non-rigid 3D motion, shape and reflectance. In: Proceedings of Eighth IEEE Interna- tional Conference on Computer Vision, ICCV, vol. 2, pp. 60-67 (2001)
  4. Fischler, M.A., Bolles, R.C.: Random sample consensus: A par- adigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381-395 (1981)
  5. Gottfried, J.-M., Fehr, J., Garbe, C.: Computing range flow from multi-modal Kinect data. In: Advances in Visual Computing, vol. 6938 of Lecture Notes in Computer Science, pp. 758-767.
  6. Springer, Berlin (2011)
  7. Hartley, R.I., Zisserman, A.: Multiple View Geometry in Com- puter Vision. Cambridge University Press, Cambridge (2000)
  8. Henry, P., Krainin, M., Herbst, E., Ren, X., Fox, D.: RGB-D mapping: Using depth cameras for dense 3D modeling of indoor environments. In: RGB-D: Advanced Reasoning with Depth Cameras Workshop in conjunction with RSS (2010)
  9. Hoppe, H., DeRose, T., Duchamp, T., McDonald, J., Stuetzle, W.: Surface reconstruction from unorganized points. In: Pro- ceedings of ACM SIGGRAPH (1992)
  10. Jessen, J.B., Pilz, F., Kraft, D., Pugeault, N., Kru ¨ger, N.: Accu- mulation of different visual feature descriptors in a coherent framework. In: Scandinavian Conference on Image Analysis (SCIA) (2011)
  11. Julier, S.J., Idak Industries: The scaled unscented transformation. In: Proceedings of IEEE Amer. Control Conf, pp. 4555-4559 (2002)
  12. Khoshelham, K.: Accuracy analysis of Kinect depth data. In: ISPRS Workshop Laser Scanning, vol. XXXVIII (2011)
  13. Kim, Y.M., Chan, D., Theobalt, C., Thrun, S.: Design and cali- bration of a multi-view TOF sensor fusion system. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2008. CVPRW'08, pp. 1-7 (2008)
  14. Kim, Y.M., Chan, D., Theobalt, C., Thrun, S.: Design and cali- bration of a multi-view TOF sensor fusion system. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2008. CVPRW '08 (2008)
  15. Kitagawa, G.: Monte Carlo filter and smoother for non-gaussian nonlinear state space models. J. Comput. Graph. Stat. 5(1), 1-25 (1996)
  16. Klasing, K., Althoff, D., Wollherr, D., Buss, M.: Comparison of surface normal estimation methods for range sensing applica- tions. In: Proceedings of the 2009 IEEE International Conference on Robotics and Automation, ICRA'09, pp. 1977-1982, IEEE Press, Piscataway, NJ, USA (2009)
  17. Kramer, J., Burrus, N., Herrera D.C., Echtler, F., Parker, M.: Hacking the Kinect. Apress (2012)
  18. Middleton, L., Sivaswamy, J.: Hexagonal Image Processing : A Practical Approach. Springer, London (2005)
  19. Murray, D., Little, J.J.: Patchlets: Representing stereo vision data with surface elements. In: Seventh IEEE Workshops on Appli- cation of Computer Vision, 2005 (WACV/MOTIONS '05 ), vol. 1, pp. 192-199 (2005)
  20. Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohli, P., Shotton, J., Hodges, S., Fitzgibbon, A.: Kinectfusion: Real-time dense surface mapping and tracking. In: ISMAR '11 Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality, pp. 127-136 (2011)
  21. Pfister, H., Zwicker, M., van Baar, J., Gross, M.: Surfels: Surface elements as rendering primitives. In: ACM SIGGRAPH, pp. 335-342 (2000)
  22. Popovic ´, M., Kootstra, G., Jørgensen, J.A., Kragic, D., Kru ¨ger, N.: Grasping unknown objects using an early cognitive vision system for general scene understanding. In: Proceedings of the International Conference on Intelligent Robots and Systems (IROS), September 25-30, San Francisco, CA (2011)
  23. Pugeault, N., Kru ¨ger, N.: Temporal accumulation of oriented visual features. J. Vis. Commun. Image Represent. 22(2), 153-163 (2011)
  24. Quigley, M., Gerkey, B., Conley, K., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R., Ng, A.: ROS: An open-source Robot Operating System. In: ICRA Workshop on Open Source Software
  25. Reynolds, M., Dobos, J., Peel, L., Weyrich, T., Brostow, G.J.: Capturing time-of-flight data with confidence. In: IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR) (2011)
  26. Rusu, R.B., Cousins, S.: 3D is here: Point Cloud Library (PCL). In: IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, May 9-13 2011
  27. Saenko, K., Karayev, S., Jia, Y., Shyr, A., Janoch, A., Long, J., Fritz, M., Darrell, T.: Practical 3-D object detection using cate- gory and instance-level appearance models. In: IROS (2011)
  28. J Real-Time Image Proc
  29. Schuon, S., Theobalt, C., Davis, J., Thrun, S.: LidarBoost: superresolution for ToF 3D shape scanning. In: Proceedings of IEEE CVPR (2009)
  30. Smisek, J., Jancosek, M., Pajdla, T.: 3D with Kinect. In: IEEE Workshop on Consumer Depth Cameras for Computer Vision, November 2011
  31. Xia, L., Chen, C.-C, Aggarwal, J.K.: Human detection using depth information by Kinect. In: International Workshop on Human Activity Understanding from 3D Data in conjunction with CVPR (HAU3D), June 2011