research-article

Learning 3D mesh segmentation and labeling

Authors:

Evangelos Kalogerakis,

Aaron Hertzmann,

Karan SinghAuthors Info & Claims

SIGGRAPH '10: ACM SIGGRAPH 2010 papers

Article No.: 102, Pages 1 - 12

https://doi.org/10.1145/1833349.1778839

Published: 26 July 2010 Publication History

Abstract

This paper presents a data-driven approach to simultaneous segmentation and labeling of parts in 3D meshes. An objective function is formulated as a Conditional Random Field model, with terms assessing the consistency of faces with labels, and terms between labels of neighboring faces. The objective function is learned from a collection of labeled training meshes. The algorithm uses hundreds of geometric and contextual label features and learns different types of segmentations for different tasks, without requiring manual parameter tuning. Our algorithm achieves a significant improvement in results over the state-of-the-art when evaluated on the Princeton Segmentation Benchmark, often producing segmentations and labelings comparable to those produced by humans.

Supplementary Material

MP4 File (tp117-10.mp4)

Download
32.16 MB

References

[1]

Anguelov, D., Taskar, B., Chatalbashev, V., Koller, D., Gupta, D., Heitz, G., and Ng, A. 2005. Discriminative Learning of Markov Random Fields for Segmentation of 3D Scan Data. In CVPR.

Digital Library

[2]

Attene, M., Katz, S., Mortara, M., Patane, G., Spagnuolo, M., and Tal, A. 2006. Mesh Segmentation - A Comparative Study. In Proc. SMI.

Digital Library

[3]

Attene, M., Falcidieno, B., and Spagnuolo, M. 2006. Hierarchical Mesh Segmentation Based on Fitting Primitives. Vis. Comput. 22, 3.

Digital Library

[4]

Belongie, S., Malik, J., and Puzicha, J. 2002. Shape Matching and Object Recognition Using Shape Contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24, 4.

Digital Library

[5]

Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast Approximate Energy Minimization via Graph Cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23, 11.

Digital Library

[6]

Chen, X., Golovinskiy, A., and Funkhouser, T. 2009. A Benchmark for 3D Mesh Segmentation. ACM Trans. Graphics 28, 3.

Digital Library

[7]

Duygulu, P., Barnard, K., de Freitas, N., and Forsyth, D. 2002. Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary. In Proc. ECCV.

Digital Library

[8]

Friedman, J., Hastie, T., and Tibshirani, R. 2000. Additive Logistic Regression: a Statistical View of Boosting. The Annals of Statistics 38, 2.

[9]

Fu, H., Cohen-Or, D., Dror, G., and Sheffer, A. 2008. Upright Orientation of Man-made Objects. ACM Trans. Graph. 27, 3.

Digital Library

[10]

Gal, R., and Cohen-Or, D. 2006. Salient Geometric Features for Partial Shape Matching and Similarity. ACM Trans. Graph. 25, 1.

Digital Library

[11]

Gama, J., and Brazdil, P. 2000. Cascade Generalization. Mach. Learn. 41, 3.

Digital Library

[12]

Geman, S., and Geman, D. 1984. Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images. IEEE Trans. PAMI 6, 6, 721--741.

Digital Library

[13]

Golovinskiy, A., and Funkhouser, T. 2008. Randomized Cuts for 3D Mesh Analysis. ACM Trans. on Graph. 27, 5.

Digital Library

[14]

Golovinskiy, A., and Funkhouser, T. 2009. Consistent Segmentation of 3D Models. Proc. SMI 33, 3.

[15]

Golovinskiy, A., Kim, V. G., and Funkhouser, T. 2009. Shape-based Recognition of 3D Point Clouds in Urban Environments. In Proc. ICCV.

[16]

He, X., Zemel, R., and Carreira-Perpiñán, M. A. 2004. Multiscale Conditional Random Fields for Image Labeling. In Proc. CVPR, vol. 2.

Digital Library

[17]

Hilaga, M., Shinagawa, Y., Kohmura, T., and Kunii, T. L. 2001. Topology Matching for Fully Automatic Similarity Estimation of 3d Shapes. In SIGGRAPH.

Digital Library

[18]

Huang, Q., Wicke, M., Adams, B., and Guibas, L. 2009. Shape Decomposition Using Modal Analysis. J. Computer Graphics Forum 28.

[19]

Igarashi, T., Matsuoka, S., and Tanaka, H. 2007. Teddy: A Sketching Interface for 3d Freeform Design. In SIGGRAPH.

Digital Library

[20]

Johnson, A., and Hebert, M. 1999. Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes. IEEE Trans. PAMI 21, 5, 433--449.

Digital Library

[21]

Katz, S., and Tal, A. 2003. Hierarchical Mesh Decomposition Using Fuzzy Clustering and Cuts. ACM Trans. Graphics.

Digital Library

[22]

Katz, S., Leifman, G., and Tal, A. 2005. Mesh segmentation using feature point and core extraction. Visual Computer 21, 8.

[23]

Konishi, S., and Yuille, A. 2000. Statistical Cues for Domain Specific Image Segmentation With Performance Analysis. Proc. CVPR.

[24]

Kraevoy, V., Julius, D., and Sheffer, A. 2007. Model Composition From Interchangeable Components. In Proc. PG.

Digital Library

[25]

Kumar, S., and Hebert, M. 2003. Discriminative Random Fields: A Discriminative Framework for Contextual Interaction in Classification. In Proc. ICCV.

Digital Library

[26]

Lafferty, J. D., McCallum, A., and Pereira, F. C. N. 2001. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In ICML.

Digital Library

[27]

Lai, Y.-K., Hu, S.-M., Martin, R. R., and Rosin, P. L. 2008. Fast Mesh Segmentation Using Random Walks. In ACM symposium on Solid and Physical Modeling.

Digital Library

[28]

Lavoué, G., and Wolf, C. 2008. Markov Random Fields for Improving 3D Mesh Analysis and segmentation. In Eurographics workshop on 3D object retrieval.

Digital Library

[29]

Li, X., Gu, X., and Qin, H. 2008. Surface matching using consistent pants decomposition. In ACM Symposium on Solid and Physical Modeling.

Digital Library

[30]

Lim, E., and Suter, D. 2007. Conditional Random Field for 3D Point Clouds With Adaptive Data Reduction. In Cyberworlds.

Digital Library

[31]

Lin, H.-Y. S., Liao, H.-Y. M., and Lin, J.-C. 2007. Visual Salience-Guided Mesh Decomposition. IEEE Transactions on Multimedia 9, 1.

Digital Library

[32]

Liu, R., and Zhang, H. 2004. Segmentation of 3D Meshes Through Spectral Clustering. In Proc. PG.

Digital Library

[33]

Liu, R. F., Zhang, H., Shamir, A., and Cohen-Or, D. 2009. A Part-Aware Surface Metric for Shape Analysis. Computer Graphics Forum, (Eurographics 2009) 28, 2.

[34]

Mangan, A. P., and Whitaker, R. T. 1999. Partitioning 3D Surface Meshes Using Watershed Segmentation. IEEE Trans. on Vis. and Comp. Graph. 5, 4.

Digital Library

[35]

Munoz, D., Vandapel, N., and Hebert, M. 2008. Directional Associative Markov Network for 3-D Point Cloud Classification. In Proc. 3DPVT.

[36]

Pekelny, Y., and Gotsman, C. 2008. Articulated Object Reconstruction and Markerless Motion Capture from Depth Video. J. Computer Graphics Forum 27, 399--408.

[37]

Schnitman, Y., Caspi, Y., Cohen-or, D., and Lischinski, D. 2006. Inducing Semantic Segmentation From an Example. In Proc. ACCV.

Digital Library

[38]

Shamir, A. 2008. A Survey on Mesh Segmentation Techniques. Computer Graphics Forum 26, 6.

[39]

Shapira, L., Shalom, S., Shamir, A., Zhang, R. H., and Cohen-Or, D. In Press. Contextual Part Analogies in 3D Objects. International Journal of Computer Vision.

Digital Library

[40]

Shlafman, S., Tal, A., and Katz, S. 2002. Metamorphosis of Polyhedral Surfaces Using Decomposition. In Eurographics.

[41]

Shotton, J., Johnson, M., and Cipolla, R. 2008. Semantic Texton Forests for Image Categorization and Segmentation. In Proc. CVPR.

[42]

Shotton, J., Winn, J., Rother, C., and Criminisi, A. 2009. TextonBoost for Image Understanding: Multi-Class Object Recognition and Segmentation by Jointly Modeling Texture, Layout, and Context. Int. J. Comput. Vision 81, 1.

Digital Library

[43]

Simari, P., Kalogerakis, E., and Singh, K. 2006. Folding Meshes: Hierarchical Mesh Segmentation Based on Planar Symmetry. In SGP.

Digital Library

[44]

Simari, P., Nowrouzezahrai, D., Kalogerakis, E., and Singh, K. 2009. Multi-objective shape segmentation and labeling. Computer Graphics Forum 28, 5.

Digital Library

[45]

Torralba, A., Murphy, K. P., and Freeman, W. T. 2007. Sharing Visual Features for Multiclass and Multiview Object Detection. IEEE Trans. Pattern Anal. Mach. Intell. 29, 5.

Digital Library

[46]

Tu, Z., Chen, X., Yuille, A., and Zhu, S.-C. 2005. Image Parsing: Unifying Segmentation, Detection, and Recognition. International Journal of Computer Vision 63, 2.

Digital Library

[47]

Tu, Z. 2008. Auto-context and its Application to High-level Vision Tasks. In Proc. CVPR.

[48]

Zhang, E., Mischaikow, K., and Turk, G. 2005. Feature-based Surface Parameterization and Texture Mapping. ACM Trans. Graph. 24, 1.

Digital Library

Cited By

Kim SKim Y(2024)A Novel Method Using 3D Interest Points to Place Markers on a Large Object in Augmented RealityApplied Sciences10.3390/app1402094114:2(941)Online publication date: 22-Jan-2024
https://doi.org/10.3390/app14020941
Yuan YWu YFan XGong MMa WMiao Q(2024)EGST: Enhanced Geometric Structure Transformer for Point Cloud RegistrationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332957830:9(6222-6234)Online publication date: Sep-2024
https://doi.org/10.1109/TVCG.2023.3329578
Yuan ZXin BZhang JXu Y(2024)Three-dimensional fabric smoothness evaluation using point cloud data for enhanced quality controlJournal of Intelligent Manufacturing10.1007/s10845-024-02367-6Online publication date: 18-May-2024
https://doi.org/10.1007/s10845-024-02367-6
Show More Cited By

Index Terms

Learning 3D mesh segmentation and labeling
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Image segmentation
        Video segmentation

Recommendations

3D Mesh Labeling via Deep Convolutional Neural Networks

This article presents a novel approach for 3D mesh labeling by using deep Convolutional Neural Networks (CNNs). Many previous methods on 3D mesh labeling achieve impressive performances by using predefined geometric features. However, the generalization ...
Learning 3D mesh segmentation and labeling

This paper presents a data-driven approach to simultaneous segmentation and labeling of parts in 3D meshes. An objective function is formulated as a Conditional Random Field model, with terms assessing the consistency of faces with labels, and terms ...
Semi-supervised Mesh Segmentation and Labeling

Recently, approaches have been put forward that focus on the recognition of mesh semantic meanings. These methods usually need prior knowledge learned from training dataset, but when the size of the training dataset is small, or the meshes are too ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGGRAPH '10: ACM SIGGRAPH 2010 papers

July 2010

984 pages

ISBN:9781450302104

DOI:10.1145/1833349

Conference Chair:
Tony DeRose,
Editor:
Hugues Hoppe
ACM Transactions on Graphics

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 July 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

SIGGRAPH '10

Sponsor:

SIGGRAPH

SIGGRAPH '10: Special Interest Group on Computer Graphics and Interactive Techniques Conference

July 26 - 30, 2010

California, Los Angeles

Acceptance Rates

SIGGRAPH '10 Paper Acceptance Rate 103 of 390 submissions, 26%;

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

98
Total Citations
View Citations
5,181
Total Downloads

Downloads (Last 12 months)110
Downloads (Last 6 weeks)12

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kim SKim Y(2024)A Novel Method Using 3D Interest Points to Place Markers on a Large Object in Augmented RealityApplied Sciences10.3390/app1402094114:2(941)Online publication date: 22-Jan-2024
https://doi.org/10.3390/app14020941
Yuan YWu YFan XGong MMa WMiao Q(2024)EGST: Enhanced Geometric Structure Transformer for Point Cloud RegistrationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332957830:9(6222-6234)Online publication date: Sep-2024
https://doi.org/10.1109/TVCG.2023.3329578
Yuan ZXin BZhang JXu Y(2024)Three-dimensional fabric smoothness evaluation using point cloud data for enhanced quality controlJournal of Intelligent Manufacturing10.1007/s10845-024-02367-6Online publication date: 18-May-2024
https://doi.org/10.1007/s10845-024-02367-6
Zhang CHotta KUeshima TYu JGu C(2023)Augment with Teacher and Distill with Student: A Two-Stage Teacher-Student Network Training Scheme for 3D Human SegmentationProceedings of the 2023 12th International Conference on Computing and Pattern Recognition10.1145/3633637.3633696(375-379)Online publication date: 27-Oct-2023
https://dl.acm.org/doi/10.1145/3633637.3633696
Gao ZYan JZhai GZhang JYang X(2023)Robust Mesh Representation Learning via Efficient Local Structure-Aware Anisotropic ConvolutionIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.315160934:11(8566-8578)Online publication date: Nov-2023
https://doi.org/10.1109/TNNLS.2022.3151609
Yang FZhan LWei HChengyu LShi D(2023)Mmwave radar and vision fusion for semantic 3D reconstruction2023 IEEE 7th International Symposium on Electromagnetic Compatibility (ISEMC)10.1109/ISEMC58300.2023.10370496(1-4)Online publication date: 20-Oct-2023
https://doi.org/10.1109/ISEMC58300.2023.10370496
Mu ALiu ZDuan GTan J(2023)Part-to-Surface Mesh Segmentation for Mechanical Models Based on Multi-Stage ClusteringComputer-Aided Design10.1016/j.cad.2023.103545162:COnline publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1016/j.cad.2023.103545
Yun HKim EKim DPark HJun M(2023)Machine Learning for Object Recognition in Manufacturing ApplicationsInternational Journal of Precision Engineering and Manufacturing10.1007/s12541-022-00764-624:4(683-712)Online publication date: 16-Jan-2023
https://doi.org/10.1007/s12541-022-00764-6
Xie HZhong Y(2023)Consistent 3D human body segmentation based on combinatorial descriptor in spectral domainMultimedia Tools and Applications10.1007/s11042-023-14729-y82:18(27927-27947)Online publication date: 27-Feb-2023
https://doi.org/10.1007/s11042-023-14729-y
Wang WPan WDai CDazeley RWei LRolfe BLu X(2023)Segmentation-driven feature-preserving mesh denoisingThe Visual Computer10.1007/s00371-023-03161-w40:9(6201-6217)Online publication date: 29-Nov-2023
https://doi.org/10.1007/s00371-023-03161-w
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten