Abstract
This paper introduces an approach for automatic foreground extraction from videos utilizing depth information from time of flight(ToF) cameras. We give a clear definition of background and foreground based on 3D scene geometry and provide means of foreground extraction based on one-dimensional histograms in 3D space. Further a refinement step based on hierarchical grab-cut segmentation in a video volume with incorporated time constraints is proposed. Our approach is able to extract time-consistent foreground objects even for a moving camera and for dynamic scene content, but is limited to indoor scenarios.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Bartczak, B., Schiller, I., Beder, C., Koch, R.: Integration of a time-of-flight camera into a mixed reality system for handling dynamic scenes, moving viewpoints and occlusions in real-time. In: Proceedings of the 3DPVT Workshop (2008)
Boykov, Y., Funka-Lea, G.: Graph cuts and efficient nd image segmentation. International Journal of Computer Vision 70(2), 109–131 (2006)
Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(9), 1124–1137 (2004)
Crabb, R., Tracey, C., Puranik, A., Davis, J.: Real-time foreground segmentation via range and color imaging. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2008, pp. 1–5 (2008)
Criminisi, A., Cross, G., Blake, A., Kolmogorov, V.: Bilayer Segmentation of Live Video. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 53–60 (2006)
Fischler, M., Bolles, R.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM 24(6), 381–395 (1981)
Frick, A., Bartczak, B., Koch, R.: Real-time preview for layered depth video in 3D-TV. In: Proceedings of SPIE 7724, 77240F (2010)
Kolb, A., Barth, E., Koch, R., Larsen, R.: Time-of-flight sensors in computer graphic. In: Eurographics 2009 - State of the Art Reports pp. 119–134 (2009)
Kolmogorov, V., Criminisi, A., Blake, A., Cross, G., Rother, C.: Bi-layer segmentation of binocular stereo video. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 407–414 (2005)
Li, Y., Sun, J., Shum, H.: Video object cut and paste. ACM Transactions on Graphics (TOG)Â 24(3), 600 (2005)
Li, Y., Sun, J., Tang, C., Shum, H.: Lazy snapping. ACM Transactions on Graphics (TOG) 23(3), 303–308 (2004)
Orchard, M., Bouman, C.: Color quantization of images. IEEE Transactions on Signal Processing 39(12), 2677–2690 (1991)
Pham, V., Takahashi, K., Naemura, T.: Live video segmentation in dynamic backgrounds using thermal vision. Advances in Image and Video Technology, 143–154 (2009)
Rother, C., Kolmogorov, V., Blake, A.: Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics 23, 309–314 (2004)
Schiller, I., Beder, C., Koch, R.: Calibration of A PMD-Camera using a Planar Calibration Pattern Together with a Multi-camera Setup. In: Proc. XXXVII Int’l Soc. for Photogrammetry (2008)
Smith, A., Blinn, J.: Blue screen matting. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, pp. 259–268 (1996)
Sun, J., Zhang, W., Tang, X., Shum, H.-Y.: Background cut. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 628–641. Springer, Heidelberg (2006)
Wang, O., Finger, J., Yang, Q., Davis, J., Yang, R.: Automatic Natural Video Matting with Depth. In: 15th Pacific Conference on Computer Graphics and Applications, PG 2007, pp. 469–472 (2007)
Wu, Q., Boulanger, P., Bischof, W.: Robust Real-Time Bi-Layer Video Segmentation Using Infrared Video. In: Canadian Conference on Computer and Robot Vision, CRV 2008, pp. 87–94 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Frick, A., Franke, M., Koch, R. (2011). Time-Consistent Foreground Segmentation of Dynamic Content from Color and Depth Video. In: Mester, R., Felsberg, M. (eds) Pattern Recognition. DAGM 2011. Lecture Notes in Computer Science, vol 6835. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23123-0_30
Download citation
DOI: https://doi.org/10.1007/978-3-642-23123-0_30
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23122-3
Online ISBN: 978-3-642-23123-0
eBook Packages: Computer ScienceComputer Science (R0)