skip to main content
10.1145/3132847.3132999acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

Unsupervised Feature Selection with Joint Clustering Analysis

Published: 06 November 2017 Publication History

Abstract

Unsupervised feature selection has raised considerable interests in the past decade, due to its remarkable performance in reducing dimensionality without any prior class information. Preserving reliable locality information and achieving excellent cluster separation are two critical issues for unsupervised feature selection. However, existing methods cannot tackle two issues simultaneously. To address the problems, we propose a novel unsupervised approach that integrates sparse feature selection and robust joint clustering analysis. The joint clustering analysis seamlessly unifies the spectral clustering and the orthogonal basis clustering. Specifically, a probabilistic neighborhood graph is utilized to preserve reliable locality information in the spectral clustering, and an orthogonal basis matrix is incorporated to achieve excellent cluster separation in the orthogonal basis clustering. A compact and effective iterative algorithm is designed to optimize the proposed selection framework. Extensive experiments on both synthetic data and real-world data validate the effectiveness of our approach under various evaluation indices.

References

[1]
A. A. Alizadeh, M. B. Eisen, R. E. Davis, C. Ma, I. S. Lossos, A. Rosenwald, J. C. Boldrick, H. Sabet, T. Tran, X. Yu, et almbox. 2000. Distinct types of diffuse large B-cell lymphoma identified by gene expression profiling. Nature, Vol. 403, 6769 (2000), 503--511.
[2]
K. Bache and M. Lichman. 2013. UCI Machine Learning Repository. http://archive.ics.uci.edu/ml. (2013).
[3]
D. Cai, X. He, Y. Hu, J. Han, and T. Huang. 2007. Learning a spatially smooth subspace for face recognition Proc. 20th CVPR. 1--7.
[4]
D. Cai, C. Zhang, and X. He. 2010. Unsupervised feature selection for multi-cluster data Proc. 16th SIGKDD. 333--342.
[5]
M. Craven, A. McCallum, D. PiPasquo, T. Mitchell, and D. Freitag. 1998. Learning to extract symbolic knowledge from the World Wide Web. Coa. Mana., Vol. volume 31, 2 (1998), 121--126.
[6]
L. Du and Y. Shen. 2015. Unsupervised feature selection with adaptive structure learning Proc. 21st SIGKDD. 209--218.
[7]
S. Du, Y. Ma, S. Li, and Y. Ma. 2017. Robust unsupervised feature selection via matrix factorization. Neurocomput. Vol. 241 (2017), 115--127.
[8]
J. Dy and C. Brodley. 2004. Feature selection for unsupervised learning. J. Mach. Learn. Res., Vol. 5, Aug (2004), 845--889.
[9]
M. A. Fanty and R. A. Cole. 1990. Spoken Letter Recognition. In Proc. NIPS 3. 220--226.
[10]
I. Guyon and A. Elisseeff. 2003. An introduction to variable and feature selection. J. Mach. Learn. Res., Vol. 3, Mar (2003), 1157--1182.
[11]
D. Han and J. Kim. 2015. Unsupervised simultaneous orthogonal basis clustering feature selection Proc. 28th CVPR. 5016--5023.
[12]
X. He, D. Cai, and P. Niyogi. 2005. Laplacian score for feature selection. In Proc. NIPS 18. 507--514.
[13]
C. Hou, F. Nie, X. Li, D. Yi, and Y. Wu. 2014. Joint embedding learning and sparse regression: A framework for unsupervised feature selection. IEEE Trans. on Cyber. Vol. 44, 6 (2014), 793--804.
[14]
R. Kohavi and G. H. John. 1997. Wrappers for feature subset selection. Artif. Intell., Vol. 97, 1--2 (1997), 273--324.
[15]
D. Koller and M. Sahami. 1996. Toward optimal feature selection. In Proc. 13rd ICML. 284--292.
[16]
W. Krzanowski. 1987. Selection of variables to preserve multivariate data structure, using principal components. Appl. Statis (1987), 22--33.
[17]
P. Langley et almbox. 1994. Selection of relevant features in machine learning Proc. 8th AAAI. 245--271.
[18]
D. Lee and H. S. Seung. 2001. Algorithms for non-negative matrix factorization. Proc. NIPS 14. 556--562.
[19]
J. Li and H. Liu. 2017. Challenges of feature selection for big data analytics. IEEE Intell. Sys., Vol. 32, 2 (2017), 9--15.
[20]
Z. Li, J. Liu, Y. Yang, X. Zhou, and H. Lu. 2014. Clustering-guided sparse structural learning for unsupervised feature selection. IEEE Trans. on Knowl. Data Eng. Vol. 26, 9 (2014), 2138--2150.
[21]
Z. Li, Y. Yang, J. Liu, X. Zhou, and H. Lu. 2012. Unsupervised feature selection using nonnegative spectral analysis Proc. 26th AAAI. 1026--1032.
[22]
H. Liu, E. Dougherty, J. Dy, K. Torkkola, E. Tuv, H. Peng, C. Ding, F. Long, M. Berens, L. Parsons, et almbox. 2005. Evolving feature selection. IEEE Intell. Sys., Vol. 20, 6 (2005), 64--76.
[23]
H. Liu and L. Yu. 2005. Toward integrating feature selection algorithms for classification and clustering. IEEE Trans. on Knowl. Data Eng. Vol. 17, 4 (2005), 491--502.
[24]
P. Mitra, C. Murthy, and S. Pal. 2002. Unsupervised feature selection using feature similarity. IEEE Trans. Pattern Anal. Mach. Intell. Vol. 24, 3 (2002), 301--312.
[25]
F. Nie, H. Huang, X. Cai, and C. Ding. 2010. Efficient and robust feature selection via joint l2, 1-norms minimization Proc. NIPS 23. 1813--1821.
[26]
F. Nie, X. Wang, and H. Huang. 2014. Clustering and projected clustering with adaptive neighbors Proc. 20th SIGKDD. 977--986.
[27]
F. Nie, W. Zhu, and X. Li. 2016. Unsupervised feature selection with structured graph optimization Proc. 30th AAAI. 1302--1308.
[28]
M. Qian and C. Zhai. 2013. Robust unsupervised feature selection. In Proc. 24th IJCAI. 1621--1627.
[29]
J. Shi and J. Malik. 2000. Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. Vol. 22, 8 (2000), 888--905.
[30]
Y. Sun, S. Todorovic, and S. Goodison. 2010. Local-learning-based feature selection for high-dimensional data analysis. IEEE Trans. Pattern Anal. Mach. Intell. Vol. 32, 9 (2010), 1610--1626.
[31]
G. Valentini and T. G. Dietterich. 2003. Low bias bagged support vector machines. In Proc. 20th ICML. 752--759.
[32]
S. Wang, J. Tang, and H. Liu. 2015. Embedded Unsupervised Feature Selection. In Proc. 29th AAAI. 470--476.
[33]
W. Wang and M. A. Carreira-Perpinán. 2013. Projection onto the probability simplex: An efficient algorithm with a simple proof, and an application. Mathematics (2013), 1309--1313.
[34]
L. Wolf and A. Shashua. 2005. Feature selection for unsupervised and supervised inference: The emergence of sparsity in a weight-based approach. J. Mach. Learn. Res., Vol. 6, Nov (2005), 1855--1887.
[35]
Y. Yang, H. Shen, Z. Ma, Z. Huang, and X. Zhou. 2011. l2, 1-norm regularized discriminative feature selection for unsupervised learning Proc. 22nd IJCAI. 1589--1594.
[36]
H. Zeng and Y. Cheung. 2011. Feature selection and kernel learning for local learning-based clustering. IEEE Trans. Pattern Anal. Mach. Intell. Vol. 33, 8 (2011), 1532--1547.
[37]
D. Zhang, Z. H. Zhou, and S. Chen. 2007. Semi-supervised dimensionality reduction. In Proc. 7th ICDM. 629--634.
[38]
Z. Zhao and H. Liu. 2007. Spectral feature selection for supervised and unsupervised learning Proc. 24th ICML. 1151--1157.
[39]
Z. H. Zhou and M. Li. 2005. Tri-training: Exploiting unlabeled data using three classifiers. IEEE Trans. on Knowl. Data Eng. Vol. 17, 11 (2005), 1529--1541.
[40]
P. Zhu, Q. Hu, C. Zhang, and W. Zuo. 2016. Coupled Dictionary Learning for Unsupervised Feature Selection Proc. 30th AAAI. 2422--2428.

Cited By

View all

Index Terms

  1. Unsupervised Feature Selection with Joint Clustering Analysis

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management
    November 2017
    2604 pages
    ISBN:9781450349185
    DOI:10.1145/3132847
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 06 November 2017

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. cluster separation
    2. joint clustering analysis
    3. locality preserving
    4. unsupervised feature selection

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    CIKM '17
    Sponsor:

    Acceptance Rates

    CIKM '17 Paper Acceptance Rate 171 of 855 submissions, 20%;
    Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

    Upcoming Conference

    CIKM '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)10
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 19 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Robust sparse orthogonal basis clustering for unsupervised feature selectionExpert Systems with Applications10.1016/j.eswa.2025.126890(126890)Online publication date: Feb-2025
    • (2024)Unsupervised feature selection by learning exponential weightsPattern Recognition10.1016/j.patcog.2023.110183148(110183)Online publication date: Apr-2024
    • (2024)Spectral clustering and embedding with inter-class topology-preservingKnowledge-Based Systems10.1016/j.knosys.2023.111278284(111278)Online publication date: Jan-2024
    • (2023)Evaluation of Tropical Cyclone Disaster Loss Using Machine Learning Algorithms with an eXplainable Artificial Intelligence ApproachSustainability10.3390/su15161226115:16(12261)Online publication date: 11-Aug-2023
    • (2022)Graph Regularized Virtual Label Regression for Unsupervised Feature SelectionDigital Signal Processing10.1016/j.dsp.2022.103393(103393)Online publication date: Jan-2022
    • (2021)Semi-supervised local feature selection for data classificationScience China Information Sciences10.1007/s11432-020-3063-064:9Online publication date: 23-Aug-2021
    • (2019)Informatics Framework to Identify Consistent Diagnostic Techniques2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)10.1109/BIBM47256.2019.8983164(1481-1486)Online publication date: Nov-2019

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media