Iterative cage-based registration from multi-view silhouettes

ABSTRACT

Dynamic shape capture from casual videos is a fundamental task at the cross-fertilization of Computer Vision and Computer Graphics. Notwithstanding, recent advances in low-cost dynamic scanners turn the cross parametrization of non-rigid animatable surface into an ill-posed and vision-oriented problem. In this paper, we propose a cage-based technique to register non-rigid observed shapes using a meaningful, reduced and animator-friendly embedding. This subspace offers natural silhouette-awareness to encode the deformation complexity already encapsulated in the targets. The estimated time-varying parameters associated the underlying flexible structure allows potential reuse. In particular, we leverage the problem of highly non-rigid spacetime registration by employing an elastoplastic coarse cage. Thus, we perform scalable handle-aware biharmonic shape registration, relying on the high-level of shape abstraction offered by this space-based paradigm. Finally, we tested the effectiveness of our proposed solution on real-world datasets capturing time-varying multi-view silhouettes.

References

1. Zheng, Q., Sharf, A., Tagliasacchi, A., Chen, B., Zhang, H., Sheffer, A., and Cohen-Or, D.: Consensus skeleton for non-rigid space-time registration. Computer Graphics Forum 29, 2. (2010)
2. Xian, C., Lin, H., and Gao, S.: Automatic cage generation by improved obbs for mesh deformation. In Vis. Comput. (2012)
3. Ju, T., Zhou, Q-Y., Van de Panne, M., Cohen-Or, D. and Neumann, U.: Reusable skinning templates using cage-based deformations. In ACM Trans. Graph. (2008)
4. Ju, T., Schaefer, S. and Warren, J.: Mean value coordinates for closed triangular meshes. In ACM Trans. Graph. (2005)
5. Lipman, Y., Kopf, J., Cohen-Or, D., and Levin, D.: GPU-assisted positive mean value coordinates for mesh deformations. In SGP. (2007)
6. Lipman, Y., Levin, D., and Cohen-Or D.: Green Coordinates. In ACM Trans. Graph. (2008)
7. Joshi, P., Meyer, M., DeRose, T., Green, B., and Sanocki, T.: Harmonic coordinates for character articulation. In ACM Trans. Graph. (2007)
8. Ben-Chen, M., Weber, O., and Gotsman, C.: Variational harmonic maps for space deformation. In ACM Trans. Graph. (2009)
9. Weber O., Poranne R., and Gotsman, C.: Spatial deformation transfer, In Computer Graphics Forum. (2012)
10. Jacobson, A., Baran, I., Popović, J. and Sorkine, O.: Bounded Biharmonic Weights for Real-Time Deformation. In ACM Trans. Graph. (2011)
11. Chen, L., Huang, J., Sun, H., and Bao, H.: Cage-based deformation transfer. In Comput. Graph. (2010)
12. Ben-Chen, M., Weber, O., and Gotsman, C.: Spatial deformation transfer, In SCA. (2009)
13. Chang, W., and Zwicker, M.: Global Registration of Dynamic Range Scans for Articulated Model Reconstruction, In ACM Trans. Graph. (2011)
14. de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H-P., and Thrun, S.: Performance capture from sparse multi-view video. In SIGGRAPH'08. (2008)
15. Ballan, L., and Cortelazzo, G. M.: Marker-less motion capture of skinned models in a four camera set-up using optical flow and silhouettes. In 3DPVT. (2008)
16. Amberg, B., Romdhani, S., and Vetter, T.: Optimal Step Nonrigid ICP Algorithms for Surface Registration. In CVPR. (2007)
17. Gall, J., Stoll, C., de Aguiar, E., Theobalt, C., Rosenhahn, B., and Seidel, H.-P.: Motion Capture Using Joint Skeleton Tracking and Surface Estimation. In CVPR. (2009)
18. Budd, C., Huang, P., and Hilton, A.: Hierarchical Shape Matching for Temporally Consistent 3D Video. In 3DIMPVT. (2011)
19. Guennebaud, G., and Gross, M.: Algebraic point set surfaces. In ACM Trans. Graph. (2007)
20. Besl, P., and McKay, N.: A Method for Registration of 3-D Shapes. In TPAMI. (1992)
21. Chang, W., and Zwicker, M.: Automatic Registration for Articulated Shapes. In SGP. (2008)
22. Connor, M., and Kumar, P.: Fast Construction of k-Nearest Neighbor Graphs for Point Clouds. In TGCV. (2010)
23. Hornung, A., and Kobbelt, L.: Robust reconstruction of watertight 3D models from non-uniformly sampled point clouds without normal information. In SGP. (2006)
24. Huang, P., Budd, C., and Hilton, A.: Global temporal registration of multiple non-rigid surface sequences. In CVPR. (2011)
25. Huang, Q-X., Adams, B., Wicke, M., and Guibas, L.: Non-Rigid Registration Under Isometric Deformations. In Comput. Graph. Forum. (2008)
26. Laurentini, A.: The Visual Hull Concept for Silhouette-Based Image Understanding. In IEEE TPAMI. (1994)
27. Kim, K., Sakamoto, R., Kitahara, I., Toriyama, T., and Kogure, K.: Robust Foreground Segmentation from Color Video Sequences Using Background Subtraction with Multiple Thresholds. In Third Korea-Japan Joint Workshop on Pattern Recognition. (2006)
28. Starck, J., and Hilton, A.: Surface Capture for Performance-Based Animation. In IEEE Comput. Graph. Appl. (2007)
29. Li, H., Sumner, R., and Pauly, M.: Global Correspondence Optimization for Non-Rigid Registration of Depth Scans, In SGP. (2008)
30. Li, H., Adams, B., Guibas, L., and Pauly, M.: Robust Single-View Geometry And Motion Reconstruction. In ACM Trans. on Graph. (2009)
31. Li, H., Luo, L., Vlasic, D., Peers, P., Popović, J., Pauly, M., and Rusinkiewicz, S.: Temporally Coherent Completion of Dynamic Shapes. In ACM Trans. Graph. (2012)
32. Grauman, K., Shakhnarovich, G., and Darrell, T.: A bayesian approach to image-based visual hull reconstruction. In CVPR. (2003)
33. Kolev, K., Brox, T., and Cremers, D.: Fast Joint Estimation of Silhouettes and Dense 3D Geometry from Multiple Images. In IEEE TPAMI. (2012)
34. Matusik, W., Buehler, C., and McMillan, L.: Polyhedral Visual Hulls for Real-Time Rendering. In Eurographics Workshop on Rendering Techniques. (2001)
35. Corazza, S., Mündermann, L., Gambaretto, E., Ferrigno, G., and Andriacchi, T.: Markerless Motion Capture through Visual Hull, Articulated ICP and Subject Specific Model Generation, In Int. J. Comput. Vision. (2010)
36. Liu, Y., Stoll, C., Gall, J., Seidel, H.-P. and Theobalt, C.: Markerless motion capture of interacting characters using multi-view image segmentation In CVPR. (2011)
37. Budd, C., and Hilton, A.: Temporal Alignment of 3D Video Sequences Using Shape and Appearance. 9th Conference on Visual Media Production (CVMP). (2010)
38. Botsch, M., and Sorkine, O.: On Linear Variational Surface Deformation Methods. In TVCG. (2008)
39. Desbrun, M., and Meyer, M., Schröder, P., and Barr, A.: Implicit fairing of irregular meshes using diffusion and curvature flow In SIGGRAPH'99. (1999)
40. Wald, I., Boulos, S., and Shirley, P.: Ray tracing deformable scenes using dynamic bounding volume hierarchies. In ACM Trans. Graph. (2007)
41. Popa, T. South-Dickinson, I., Bradley D., Sheffer A., and Heidrich, W.: Globally Consistent Space-Time Reconstruction. In SGP. (2010)
42. Süssmuth, J., Zollhöfer, M., and Greiner, G.: Animation transplantation. In Comput. Animat. Virtual Worlds. (2010)
43. Münch, D., and Combès, B., and Prima, S.: A modified ICP algorithm for normal-guided surface registration. In Proceedings of SPIE 7623. (2010)
44. Sagawa, R., Akasaka, K., Yagi, Y., Ha, H., and Van Gool, L.: Elastic Convolved ICP for the Registration of Deformable Objects. In 3DIM. (2009)
45. Zheng, Q., Sharf, A., Tagliasacchi, A., Chen, B., Zhang, H., Sheffer, A., and Cohen-Or, D.: Consensus Skeleton for Non-Rigid Space-Time Registration. In Computer Graphics Forum. (2010)
46. Tevs, A., Berner, A., Wand, M., Ihrke, I., Bokeloh, M., Kerber, J., and Seidel, H.-P.: Animation cartography: intrinsic reconstruction of shape and motion. In ACM Trans. Graph., (2012).
47. Liu, Y., Dai, Q., and Xu, W.: A Point-Cloud-Based Multi-view Stereo Algorithm for Free-Viewpoint Video. In TVCG. (2010)
48. Sharf, A., Alcantara, D., Lewiner, T., Greif, C., Sheffer, A., Amenta, N. and Cohen-Or, D.: Space-time surface reconstruction using incompressible flow. In ACM Trans. Graph. (2008)
49. Stoll, C., Karni, Z., Rossl, C., Yamauchi, H., and Seidel, H.-P.: Template Deformation for Point Cloud Fitting. In SPBG. (2006)
50. Tong, J., Zhou, J., Liu, L., Pan Z., and Yan, H.: Scanning 3D Full Human Bodies using Kinects. In TVCG. (2012)
51. Vogiatzis, G., Torr, P., and Cipolla, R.: Multi-View Stereo via Volumetric Graph-Cuts. In CVPR. (2005)
52. Wald, I.: On fast Construction of SAH-based Bounding Volume Hierarchies. In Proceedings of the IEEE Symposium on Interactive Ray Tracing. (2007)
53. Au, O., Fu, H., Tai, C.-T., and Cohen-Or, D.: Handle-aware isolines for scalable shape editing. In ACM Trans. Graphic. (2007)
54. Vlasic, D., Baran, I., Matusik, W., and Popović, J.: Articulated mesh animation from multi-view silhouettes. In ACM Trans. Graph. (2008)
55. Wand, M., Jenke, P., Huang, Q.-X., Bokeloh, M., Guibas, L., and Schilling, A.: Reconstruction of deforming geometry from time-varying point clouds. In SGP. (2007)
56. Yeh I.-C, Lin, C.-H., Sorkine, O., and Lee, T.-Y.: Template-based 3D Model Fitting Using Dual-domain Relaxation. In IEEE TVCG. (2010)
57. Witkin, A., and Kass, M.: Spacetime constraints. In SIGGRAPH. (1988)
58. Luo, W., Yamasaki, T., and Aizawa, K.: Articulated Human Motion Capture from. Segmented Visual Hulls and Surface Reconstruction. In APSIPA. (2010)
59. Windheuser, T., Schlickewei, U., Schmidt, F., and Cremers, D.: Large-Scale Integer Linear Programming for Orientation-Preserving 3D Shape Matching. In SGP. (2011)
60. Huang, P., Budd, C., and Hilton, A.: Global temporal registration of multiple non-rigid surface sequences. In CVPR. (2011)
61. Jacobson, A., Baran, I., Popović, J., and Sorkine, O.: Bounded Biharmonic Weights for Real-Time Deformation. In ACM Trans. Graph. (2011)
62. Arya, S., Mount, D., Netanyahu, N., Silverman, R., and Wu, A.: An optimal algorithm for approximate nearest neighbor searching fixed dimensions. In J. ACM. (1998)

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Shape inference

      • Computer vision tasks

        • Video summarization

  • Computer graphics

    • Shape modeling

• Information systems

  • Information systems applications

    • Multimedia information systems