The 3D-Pitoti Dataset: A Dataset for high-resolution 3D Surface Segmentation

ABSTRACT

The development of powerful 3D scanning hardware and reconstruction algorithms has strongly promoted the generation of 3D surface reconstructions in different domains. An area of special interest for such 3D reconstructions is the cultural heritage domain, where surface reconstructions are generated to digitally preserve historical artifacts. While reconstruction quality nowadays is sufficient in many cases, the robust analysis (e.g. segmentation, matching, and classification) of reconstructed 3D data is still an open topic. In this paper, we target the automatic segmentation of high-resolution 3D surface reconstructions of petroglyphs. To foster research in this field, we introduce a fully annotated, large-scale 3D surface dataset including high-resolution meshes, depth maps and point clouds as a novel benchmark dataset, which we make publicly available. Additionally, we provide baseline results for a random forest as well as a convolutional neural network based approach. Results show the complementary strengths and weaknesses of both approaches and point out that the provided dataset represents an open challenge for future research.

References

1. I. Armeni, Alexander. Sax, A. R Zamir, and S. Savarese. 2017. Joint 2D-3D-Semantic Data for Indoor Scene Understanding. arXiv preprint arXiv:1702.01105 (2017).
2. L. Blunt and X. Jiang. 2003. Advanced techniques for assessment surface topography: development of a basis for 3D surface texture standards "surfstand". Elsevier.
3. Leo Breiman. 2001. Random Forests. Machine Learning 45, 1 (2001), 5--32.
4. Samuel Rota Bulò and Peter Kontschieder. 2014. Neural Decision Forests for Semantic Image Labelling. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition.
5. Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. 2015. Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs. In Proc. Int'l Conf. on Learning Representations.
6. Kristin J. Dana, Bram van Ginneken, Shree K. Nayar, and Jan J. Koenderink. 1999. Reflectance and Texture of Real-World Surfaces. ACM Trans. on Graphics 18, 1 (1999), 1--34.
7. Michael Firman. 2016. RGBD Datasets: Past, Present and Future. In CVPR Workshop on Large Scale 3D Data: Acquisition, Modelling and Analysis.
8. M. Haindl and S. Mikeš. 2008. Texture Segmentation Benchmark. In Proc. Int'l Conf. on Pattern Recognition.
9. Robert M. Haralick, K. Sam Shanmugam, and Its'hak Dinstein. 1973. Textural Features for Image Classification. IEEE Trans. Systems, Man, and Cybernetics 3, 6 (1973), 610--621.
10. Bharath Hariharan, Pablo Andrés Arbeláez, Ross B. Girshick, and Jitendra Malik. 2015. Hypercolumns for object segmentation and fine-grained localization. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition.
11. Paul Jaccard. 1912. The distribution of the flora in the alpine zone. New Phytologist 11, 2 (1912), 37--50.
12. Allison Janoch, Sergey Karayev, Yangqing Jia, Jonathan T. Barron, Mario Fritz, Kate Saenko, and Trevor Darrell. 2011. A category-level 3-D object dataset: Putting the Kinect to work. In ICCV Workshop on Consumer Depth Cameras for Computer Vision.
13. Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional Architecture for Fast Feature Embedding. arXiv preprint arXiv:1408.5093 (2014).
14. Peter Kontschieder, Samuel Rota Bulò, Marcello Pelillo, and Horst Bischof. 2014. Structured Labels in Random Forests for Semantic Labelling and Object Detection. IEEE Trans. on Pattern Analysis and Machine Intelligence 36, 10 (2014), 2104--2116.
15. Peter Kontschieder, Pushmeet Kohli, Jamie Shotton, and Antonio Criminisi. 2013. GeoF: Geodesic Forests for Learning Coupled Predictors. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition.
16. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems.
17. Dmitry Laptev and Joachim M. Buhmann. 2014. Convolutional Decision Trees for Feature Learning and Segmentation. In Proc. German Conf. on Pattern Recognition.
18. Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278--2324.
19. Guosheng Lin, Chunhua Shen, Anton van dan Hengel, and Ian Reid. 2016. Efficient piecewise training of deep structured models for semantic segmentation. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition.
20. Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition.
21. Roozbeh Mottaghi, Xianjie Chen, Xiaobai Liu, Nam-Gyu Cho, Seong-Whan Lee, Sanja Fidler, Raquel Urtasun, and Alan L. Yuille. 2014. The Role of Context for Object Detection and Semantic Segmentation in the Wild. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition.
22. Pushmeet Kohli Nathan Silberman, Derek Hoiem and Rob Fergus. 2012. Indoor Segmentation and Support Inference from RGBD Images. In Proc. European Conf. on Computer Vision.
23. Timo Ojala, Topi Mäenpää, Matti Pietikäinen, Jaakko Viertola, Juha Kyllönen, and Sami Huovinen. 2002. Outex- New Framework for Empirical Evaluation of Texture Analysis Algorithms. In Proc. Int'l Conf. on Pattern Recognition.
24. Timo Ojala and Matti Pietikäinen. 1999. Unsupervised texture segmentation using feature distributions. Pattern Recognition 32, 3 (1999), 477--486.
25. Radu Bogdan Rusu, Zoltan Csaba Marton, Nico Blodow, Mihai Dolha, and Michael Beetz. 2008. Towards 3D point cloud based object maps for household environments. Robotics and Autonomous Systems 56, 11 (2008), 927--941.
26. Evan Shelhamer, Jonathan Long, and Trevor Darrell. 2016. Fully Convolutional Networks for Semantic Segmentation. IEEE Trans. on Pattern Analysis and Machine Intelligence PP, 99 (2016), 1--12.
27. Nathan Silberman and Rob Fergus. 2011. Indoor scene segmentation using a structured light sensor. In ICCV Workshop on 3D Representation and Recognition.
28. Shuran Song, Samuel P. Lichtenberg, and Jianxiong Xiao. 2015. SUN RGB-D: A RGB-D scene understanding benchmark suite. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition.
29. Changchang Wu. 2013. Towards Linear-Time Incremental Structure from Motion. In 3D Vision - 3DV 2013, 2013 International Conference on. 127--134.
30. Matthias Zeppelzauer, Georg Poier, Markus Seidl, Christian Reinbacher, Christian Breiteneder, Horst Bischof, and Samuel Schulter. 2015. Interactive Segmentation of Rock-Art in High-Resolution 3D Reconstructions. In Proc. Int'l Conf. on Digital Heritage.
31. Matthias Zeppelzauer, Georg Poier, Markus Seidl, Christian Reinbacher, Samuel Schulter, Christian Breiteneder, and Horst Bischof. 2016. Interactive 3D Segmentation of Rock-Art by Enhanced Depth Maps and Gradient Preserving Regularization. Journal on Computing and Cultural Heritage (JOCCH) 9, 4 (2016), 19.
32. Matthias Zeppelzauer and Markus Seidl. 2015. Efficient image-space extraction and representation of 3D surface topography. In Proc. Int'l Conf. on Image Processing.
33. Shuai Zheng, Sadeep Jayasumana, Bernardino Romera-Paredes, Vibhav Vineet, Zhizhong Su, Dalong Du, Chang Huang, and Philip Torr. 2015. Conditional Random Fields as Recurrent Neural Networks. In Proc. IEEE Int'l Conf. on Computer Vision.

Index Terms

• Applied computing

  • Physical sciences and engineering

    • Archaeology

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Image segmentation