Non-local scan consolidation for 3D urban scenes

ABSTRACT

Recent advances in scanning technologies, in particular devices that extract depth through active sensing, allow fast scanning of urban scenes. Such rapid acquisition incurs imperfections: large regions remain missing, significant variation in sampling density is common, and the data is often corrupted with noise and outliers. However, buildings often exhibit large scale repetitions and self-similarities. Detecting, extracting, and utilizing such large scale repetitions provide powerful means to consolidate the imperfect data. Our key observation is that the same geometry, when scanned multiple times over reoccurrences of instances, allow application of a simple yet effective non-local filtering. The multiplicity of the geometry is fused together and projected to a base-geometry defined by clustering corresponding surfaces. Denoising is applied by separating the process into off-plane and in-plane phases. We show that the consolidation of the reoccurrences provides robust denoising and allow reliable completion of missing parts. We present evaluation results of the algorithm on several LiDAR scans of buildings of varying complexity and styles.

References

1. Aiger, D., Mitra, N. J., and Cohen-Or, D. 2008. 4-points congruent sets for robust surface registration. Proc. of ACM SIGGRAPH 27, 3, #85, 1--10.
2. Besl, P., and McKay, N. 1992. A method for registration of 3D. In IEEE PAMI.
3. Biber, P., and Strasser, W. 2006. nscan-matching: Simultaneous matching of multiple scans and application to slam. In In Robotics and Automation, 2270--2276.
4. Bokeloh, M., Berner, A., Wand, M., Seidel, H.-P., and Schilling, A. 2009. Symmetry detection using line features. Computer Graphics Forum (Proceedings of Eurographics).
5. Buades, A., Coll, B., and Morel, J.-M. 2005. A non-local algorithm for image denoising. In Proc. of IEEE Conf. on Comp. Vis. and Pat. Rec., 60--65.
6. Buades, A., Coll, B., and Morel, J.-M. 2008. Nonlocal image and movie denoising. Int. J. Comp. Vis. 76, 2, 123--139.
7. Dabov, Foi, Katkovnik, and Egiazarian. 2007. Image denoising by sparse 3-d transform-domain collaborative filtering. Image Processing, IEEE Transactions on 16, 8, 2080--2095.
8. Debevec, P. E., Taylor, C. J., and Malik, J. 1996. Modeling and rendering arch. from photographs: A hybrid geometry- and image-based approach. Proc. SIGGRAPH 30, 11--20.
9. Fleishman, S., Drori, I., and Cohen-Or, D. 2003. Bilateral mesh denoising. Proc. of ACM SIGGRAPH 22, 3, 950--953.
10. Gal, R., Shamir, A., Hassner, T., Pauly, M., and Cohen-Or, D. 2007. Surface reconstruction using local shape priors. In Proc. of Eurographics Symp. on Geometry Processing, 253--262.
11. Hays, J. H., Leordeanu, M., Efros, A. A., and Liu, Y. 2006. Discovering texture regularity as a higher-order correspondence problem. In Proc. Euro. Conf. on Comp. Vis.
12. Huang, H., Li, D., Zhang, H., Ascher, U., and Cohen-Or, D. 2009. Consolidation of unorganized point clouds for surface reconstruction. ACM Trans. Graph. 28, 5, Article 176.
13. Jones, T. R., Durand, F., and Desbrun, M. 2003. Noniterative, feature-preserving mesh smoothing. In Proc. of ACM SIGGRAPH, 943--949.
14. Korah, T., and Rasmussen, C. 2007. 2d lattice extraction from structured environments. In ICIP, 61--64.
15. Korah, T., and Rasmussen, C. 2008. Analysis of building textures for reconstructing partially occluded facades. In Proc. Euro. Conf. on Comp. Vis., 359--372.
16. Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginzton, M., Anderson, S., Davis, J., Ginsberg, J., Shade, J., and Fulk, D. 2000. The digital michelangelo project: 3d scanning of large statues. In Proc. of ACM SIGGRAPH, 131--144.
17. Lindenbaum, M., Fischer, M., and Bruckstein, A. M. 1994. On gabor's contribution to image enhancement. Pattern Recognition 27, 1, 1--8.
18. Lipman, Y., Cohen-Or, D., Levin, D., and Tal-Ezer, H. 2007. Parameterization-free projection for geometry reconstruction. ACM Trans. Graph. 26, 3, 22.
19. Liu, Y., Collins, R. T., and Tsin, Y. 2004. A computational model for periodic pattern perception based on frieze and wallpaper groups. IEEE PAMI 26, 3, 354--371.
20. Liu, Y., Lin, W.-C., and Hays, J. H. 2004. Near regular texture analysis and manipulation. 368--376.
21. Merritt, F., and Ricketts, J. 2001. Building Design and Construction Handbook, 6th ed. McGraw-Hill.
22. Mitra, N. J., Guibas, L., and Pauly, M. 2006. Partial and approximate symmetry detection for 3d geometry. In Proc. of ACM SIGGRAPH, vol. 25, 560--568.
23. Müller, P., Zeng, G., Wonka, P., and Gool, L. J. V. 2007. Image-based procedural modeling of facades. ACM Trans. on Graphics 26, 3, 85.
24. Musialski, P., Wonka, P., Recheis, M., Maierhofer, S., and Purgathofer, W. 2009. Symmetry-based facade repair. In Vision, Modeling, and Visualization Workshop 2009 in Braunschweig, Germany (VMV09).
25. Oztireli, C., Guennebaud, G., and Gross, M. 2009. Feature preserving point set surfaces based on non-linear kernel regression. Proc. Eurographics) 28, 2, 493--501.
26. Park, M., Brocklehurst, K., Collins, R. T., and Liu, Y. 2009. Deformed lattice detection in real-world images using mean-shift belief propagation. IEEE PAMI 31.
27. Pauly, M., Mitra, N. J., Giesen, J., Gross, M., and Guibas, L. 2005. Example-based 3d scan completion. In Proc. of Symp. of Geometry Processing, 23--32.
28. Pauly, M., Mitra, N. J., Wallner, J., Pottmann, H., and Guibas, L. 2008. Discovering structural regularity in 3D geometry. ACM Trans. on Graphics 27, 3.
29. Perona, P., and Malik, J. 1990. Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12, 7, 629--639.
30. Rudin, L. I., Osher, S., and Fatemi, E. 1992. Nonlinear total variation based noise removal algorithms. Physica D 60, 1--4, 259--268.
31. Schaffalitzky, F., and Zisserman, A. 1999. Geometric grouping of repeated elements within images. In Shape, Contour and Grouping in Computer Vision, 165--181.
32. Schnabel, R., Degener, P., and Klein, R. 2009. Completion and reconstruction with primitive shapes. Computer Graphics Forum (Proc. of Eurographics) 28, 2, 503--512.
33. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proc. of Int. Conf. on Comp. Vis., 839.
34. Wang, X., Totaro, S., Taill, F., Hanson, A. R., and Teller, S. 2002. Recovering facade texture and microstructure from real-world images. In Texture Analysis and Synth., 381--386.
35. Xiao, J., Fang, T., Zhao, P., Lhuillier, M., and Quan, L. 2009. Image-based street-side city modeling. In ACM SIGGRAPH Asia 2009 papers, 1--12.
36. Yoshizawa, S., Belyaev, A., and Seidel, H.-P. 2006. Smoothing by example: Mesh denoising by averaging with similarity-based weights. In SMI, 38--44.
37. Yu, Y., Ferencz, A., and Malik, J. 2001. Extracting objects from range and radiance images. IEEE Transactions on Visualization and Computer Graphics 7, 4, 351--364.

Index Terms

• Applied computing

  • Document management and text processing

    • Document capture

      • Document scanning

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Image and video acquisition

  • Machine learning

    • Learning paradigms

      • Unsupervised learning

        • Cluster analysis

• Hardware

  • Communication hardware, interfaces and storage

    • Signal processing systems