Abstract
In this paper, we tackle the problem of finding correspondences between three-dimensional reconstructions of a deformable surface at different time steps. We suppose that (i) the mechanical underlying model imposes time-constant geodesic distances between points on the surface; and that (ii) images of the real surface are available. This is for instance the case in spatio-temporal shape from videos (e.g. multi-view stereo, visual hulls, etc.) when the surface is supposed approximatively unstretchable. These assumptions allow to exploit both geometry and photometry. In particular we propose an energy based formulation of the problem, extending the work of Bronstein et al. [1]. On the one hand, we show that photometry (i) improves accuracy in case of locally elastic deformations or noisy surfaces and (ii) allows to still find the right solution when [1] fails because of ambiguities (e.g. symmetries). On the other hand, using geometry makes it possible to match shapes that have undergone large motion, which is not possible with usual photometric methods. Numerical experiments prove the efficiency of our method on synthetic and real data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Bronstein, A., Bronstein, M., Kimmel, R.: Generalized multidimensional scaling: a framework for isometry-invariant partial surface matching. Proc. National Academy of Sciences (PNAS) 103(5), 1168–1172 (2006)
Vedula, S., Baker, S., Rander, P., Collins, R., Kanade, T.: Three-dimensional scene flow (2005)
Pons, J.P., Keriven, R., Faugeras, O.: Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. The International Journal of Computer Vision 72(2), 179–193 (2007)
Gelfand, N., Mitra, N.J., Guibas, L.J., Pottmann, H.: Robust global registration. In: Desbrun, M., Pottmann, H. (eds.) Eurographics Association, pp. 197–206 (2005) ISBN 3-905673-24-X
Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 509–522 (2002)
Amberg, B., Romdhani, S., Vetter, T.: Optimal step nonrigid icp algorithms for surface registration. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2007)
Brown, B., Rusinkiewicz, S.: Global non-rigid alignment of 3-D scans. ACM Transactions on Graphics (Proc. SIGGRAPH) 26(3) (August 2007)
Elad, A., Kimmel, R.: On bending invariant signatures for surfaces. In: IEEE Trans. on Pattern Analysis and Machine Intell. LNCS, vol. 25, pp. 1285–1295. Springer, Heidelberg (2003)
Bronstein, A., Bronstein, M., Kimmel, R.: Efficient computation of isometry-invariant distances between surfaces. SIAM J. Sci. Comput. 28 (2006)
Furukawa, Y., Ponce, J.: Dense 3d motion capture from synchronized video streams. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Los Alamitos (2008)
Huguet, F., Devernay, F.: A variational method for scene flow estimation from stereo sequences. In: ICCV, pp. 1–7 (2007)
Starck, J., Hilton, A.: Correspondence labelling for wide-timeframe free-form surface matching. In: IEEE International Conference on Computer Vision, ICCV (2007)
Starck, J., Hilton, A.: Spherical matching for temporal correspondence of non-rigid surfaces. In: ICCV, pp. 1387–1394 (2005)
Ahmed, N., Theobalt, C., Roessl, C., Thrun, S., Seidel: Dense correspondence finding for parametrization-free animation reconstruction from video. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Los Alamitos (2008)
Varanasi, K., Zaharescu, A., Boyer, E., Horaud, R.: Temporal surface tracking using mesh evolution. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 30–43. Springer, Heidelberg (2008)
Keriven, R., Faugeras, O.: Complete dense stereovision using level set methods. In: Burkhardt, H.-J., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1406, p. 379. Springer, Heidelberg (1998)
Allène, C., Pons, J.P., Keriven, R.: Seamless image-based texture atlases using multi-band blending. In: 19th International Conference on Pattern Recognition, Tampa, US (December 2008)
Sethian, J.A.: Fast marching methods. SIAM Review 41, 199–235 (1999)
Kimmel, R., Sethian, J.: Computing geodesic paths on manifolds (1998)
Bronstein, M., Bronstein, A., Kimmel, R.: Efficient computation of isometry-invariant distances between surfaces. Techn. Report CIS-2006-02, Dept. of Computer Science, Technion, Israel (2006)
Moenning, C., Dodgson, N.A.: Fast marching farthest point sampling. In: EuroGraphics (September 2003)
Dasgupta, S., Papadimitriou, C., Vazirani, U.: Algorithms. McGrawHill, New York (2007)
Starck, J., Hilton, A.: Surface capture for performance based animation. IEEE Computer Graphics and Applications 27(3), 21–31 (2007)
Bronstein, A., Bronstein, M., Kimmel, R., Mahmoudi, M., Sapiro, G.: A gromov-hausdorff framework with diffusion geometry for topologically-robust non-rigid shape matching. International Journal of Computer Vision (IJCV) (submitted)
Vu, H., Keriven, R., Labatut, P., Pons, J.: Towards high-resolution large-scale multi-view stereo. In: Conference on Computer Vision and Pattern Recognition (CVPR) (June 2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Thorstensen, N., Keriven, R. (2010). Non-rigid Shape Matching Using Geometry and Photometry. In: Zha, H., Taniguchi, Ri., Maybank, S. (eds) Computer Vision – ACCV 2009. ACCV 2009. Lecture Notes in Computer Science, vol 5996. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12297-2_62
Download citation
DOI: https://doi.org/10.1007/978-3-642-12297-2_62
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12296-5
Online ISBN: 978-3-642-12297-2
eBook Packages: Computer ScienceComputer Science (R0)