Super-Trajectory for Video Segmentation (original) (raw)

Abstract

We introduce a novel semi-supervised video segmentation approach based on an efficient video representation, called as "super-trajectory". Each super-trajectory corresponds to a group of compact trajectories that exhibit consistent motion patterns, similar appearance and close spatiotemporal relationships. We generate trajectories using a probabilistic model, which handles occlusions and drifts in a robust and natural way. To reliably group trajectories, we adopt a modified version of the density peaks based clustering algorithm that allows capturing rich spatiotemporal relations among trajectories in the clustering process. The presented video representation is discriminative enough to accurately propagate the initial annotations in the first frame onto the remaining video frames. Extensive experimental analysis on challenging benchmarks demonstrate our method is capable of distinguishing the target objects from complex backgrounds and even reidentifying them after occlusions.

Loading...

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

References (51)

  1. R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Susstrunk. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE PAMI, 2012.
  2. V. Badrinarayanan, F. Galasso, and R. Cipolla. Label propa- gation in video sequences. In CVPR, 2010.
  3. X. Bai, J. Wang, D. Simons, and G. Sapiro. Video SnapCut: robust video object cutout using localized classifiers. ACM Trans. on Graphics, 2009.
  4. W. Brendel and S. Todorovic. Video object segmentation by tracking regions. In ICCV, 2009.
  5. T. Brox and J. Malik. Object segmentation by long term analysis of point trajectories. In ECCV, 2010.
  6. T. Brox and J. Malik. Large displacement optical flow: De- scriptor matching in variational motion estimation. IEEE PAMI, 2011.
  7. I. Budvytis, V. Badrinarayanan, and R. Cipolla. Semi- supervised video segmentation using tree structured graph- ical models. In CVPR, 2011.
  8. J. Chang, D. Wei, and J. W. Fisher. A video representation using temporal superpixels. In CVPR, 2013.
  9. L. Chen, J. Shen, W. Wang, and B. Ni. Video object segmen- tation via dense trajectories. IEEE TMM, 2015.
  10. A. Faktor and M. Irani. Video segmentation by non-local consensus voting. In BMVC, 2014.
  11. Q. Fan, F. Zhong, D. Lischinski, D. Cohen-Or, and B. Chen. JumpCut: Non-successive mask transfer and interpolation for video cutout. ACM Trans. on Graphics, 2015.
  12. K. Fragkiadaki, P. Arbelaez, P. Felsen, and J. Malik. Learn- ing to segment moving objects in videos. In CVPR, 2015.
  13. K. Fragkiadaki and J. Shi. Detection free tracking: Exploit- ing motion and topology for segmenting and tracking under entanglement. In CVPR, 2011.
  14. K. Fragkiadaki, G. Zhang, and J. Shi. Video segmentation by tracing discontinuities in a trajectory embedding. In CVPR, 2012.
  15. K. Fragkiadaki, W. Zhang, G. Zhang, and J. Shi. Two- granularity tracking: Mediating trajectory and detection graphs for tracking under occlusions. In ECCV, 2012.
  16. M. Grundmann, V. Kwatra, M. Han, and I. Essa. Efficient hi- erarchical graph-based video segmentation. In CVPR, 2010.
  17. B. Hariharan, P. Arbeláez, R. Girshick, and J. Malik. Hyper- columns for object segmentation and fine-grained localiza- tion. In CVPR, 2015.
  18. M. Keuper, B. Andres, and T. Brox. Motion trajectory seg- mentation via minimum cost multicuts. In ICCV, 2015.
  19. Y. J. Lee, J. Kim, and K. Grauman. Key-segments for video object segmentation. In ICCV, 2011.
  20. J. Lezama, K. Alahari, J. Sivic, and I. Laptev. Track to the future: Spatio-temporal video segmentation with long-range motion cues. In CVPR, 2011.
  21. F. Li, T. Kim, A. Humayun, D. Tsai, and J. M. Rehg. Video segmentation by tracking many figure-ground segments. In ICCV, 2013.
  22. T. Ma and L. J. Latecki. Maximum weight cliques with mutex constraints for video object segmentation. In CVPR, 2012.
  23. N. Maerki, F. Perazzi, O. Wang, and A. Sorkine-Hornung. Bilateral space video segmentation. In CVPR, 2016.
  24. P. Ochs and T. Brox. Higher order motion models and spec- tral clustering. In CVPR, 2012.
  25. P. Ochs, J. Malik, and T. Brox. Segmentation of moving objects by long term video analysis. IEEE PAMI, 2014.
  26. D. Oneata, J. Revaud, J. Verbeek, and C. Schmid. Spatio- temporal object detection proposals. In ECCV, 2014.
  27. A. Papazoglou and V. Ferrari. Fast object segmentation in unconstrained video. In ICCV, 2013.
  28. F. Perazzi, J. Pont-Tuset, B. McWilliams, L. V. Gool, M. Gross, and A. Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In CVPR, 2016.
  29. F. Perazzi, O. Wang, M. Gross, and A. Sorkinehornung. Fully connected object proposals for video segmentation. In ICCV, 2015.
  30. S. A. Ramakanth and R. V. Babu. SeamSeg: Video object segmentation using patch seams. In CVPR, 2014.
  31. A. Rodriguez and A. Laio. Clustering by fast search and find of density peaks. Science, 2014.
  32. N. Shankar Nagaraja, F. R. Schmidt, and T. Brox. Video segmentation with just a few strokes. In ICCV, 2015.
  33. J. Shi and C. Tomasi. Good features to track. In CVPR, 1994.
  34. N. Sundaram, T. Brox, and K. Keutzer. Dense point trajecto- ries by GPU-accelerated large displacement optical flow. In ECCV, 2010.
  35. D. Tsai, M. Flagg, and J. M. Rehg. Motion coherent tracking using multi-label MRF optimization. BMVC, 2010.
  36. Y.-H. Tsai, M.-H. Yang, and M. J. Black. Video segmenta- tion via object flow. In CVPR, 2016.
  37. A. Vazquez-Reina, S. Avidan, H. Pfister, and E. Miller. Mul- tiple hypothesis video segmentation from superpixel flows. In ECCV, 2010.
  38. S. Vijayanarasimhan and K. Grauman. Active frame selec- tion for label propagation in videos. In ECCV, 2012.
  39. H. Wang, A. Kläser, C. Schmid, and C.-L. Liu. Action recog- nition by dense trajectories. In CVPR, 2011.
  40. H. Wang and C. Schmid. Action recognition with improved trajectories. In ICCV, 2013.
  41. L. Wang, Y. Qiao, and X. Tang. Action recognition with trajectory-pooled deep-convolutional descriptors. In CVPR, 2015.
  42. W. Wang, J. Shen, X. Li, and F. Porikli. Robust video object cosegmentation. IEEE TIP, 2015.
  43. W. Wang, J. Shen, and F. Porikli. Saliency-aware geodesic video object segmentation. In CVPR, 2015.
  44. W. Wang, J. Shen, and L. Shao. Consistent video saliency using local gradient flow optimization and global refinement. IEEE TIP, 2015.
  45. W. Wang, J. Shen, L. Shao, and F. Porikli. Correspondence driven saliency transfer. IEEE TIP.
  46. W. Wang, J. Shen, R. Yang, and F. Porikli. Saliency-aware video object segmentation. IEEE PAMI, 2017.
  47. L. Wen, D. Du, Z. Lei, S. Z. Li, and M.-H. Yang. JOTS: Joint online tracking and segmentation. In CVPR, 2015.
  48. F. Xiao and Y. Jae Lee. Track and segment: An iterative unsupervised approach for video object proposals. In CVPR, 2016.
  49. C. Xu, C. Xiong, and J. J. Corso. Streaming hierarchical video segmentation. In ECCV, 2012.
  50. D. Zhang, O. Javed, and M. Shah. Video object segmentation through spatially accurate and temporally dense extraction of primary object regions. In CVPR, 2013.
  51. F. Zhong, X. Qin, Q. Peng, and X. Meng. Discontinuity- aware video object cutout. ACM Trans. on Graphics, 2012.