skip to main content
10.1145/2063576.2063702acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

Accounting for data dependencies within a hierarchical dirichlet process mixture model

Published: 24 October 2011 Publication History

Abstract

We propose a hierarchical nonparametric topic model, based on the hierarchical Dirichlet process (HDP), that accounts for dependencies among the data. The HDP mixture models are useful for discovering an unknown semantic structure (i.e., topics) from a set of unstructured data such as a corpus of documents. For simplicity, HDP makes an exchangeability assumption that any permutation of the data points would result in the same joint probability of the data being generated. This exchangeability assumption poses a problem for some domains where there are clear and strong dependencies among the data. A model that allows for non-exchangeability of data can capture these dependencies and assign higher probabilities to clusters that account for data dependencies, for example, inferring topics that reflect the temporal patterns of the data. Our model incorporates the distance dependent Chinese restaurant process (ddCRP), which clusters data with an inherent bias toward clusters of data points that are near to one another, into a hierarchical construction analogous to the HDP, and we call this new prior the distance dependent Chinese restaurant franchise (ddCRF). When tested with temporal datasets, the ddCRF mixture model shows clear improvements in data fit compared to the HDP in terms of heldout likelihood and complexity. The resulting set of topics shows the sequential emergence and disappearance patterns of topics.

References

[1]
D. Blei and P. Frazier. Distance dependent chinese restaurant processes. ICML, 2010.
[2]
D. Blei, A. Ng, and M. Jordan. Latent dirichlet allocation. The Journal of Machine Learning Research, pages 993--1022, Jan 2003.
[3]
L. Dietz, S. Bickel, and T. Scheffer. Unsupervised prediction of citation influences. ICML, 2007.
[4]
T. L. Griffiths, M. Steyvers, D. M. Blei, and J. B. Tenenbaum. Integrating topics and syntax. NIPS, 2005.
[5]
M. Hoffman, D. Blei, and P. Cook. Finding latent sources in recorded music with a shift-invariant hdp. DAFx, 2009.
[6]
D. Hu, X. Zhang, J. Yin, V. Zheng, and Q. Yang. Abnormal activity recognition based on hdp-hmm models. International Joint Conferences on Artificial Intelligence, 2009.
[7]
C. Lin and Y. He. Joint sentiment/topic model for sentiment analysis. CIKM, 2009.
[8]
C. Ritter and M. Tanner. Facilitating the gibbs sampler: The gibbs stopper and the griddy-gibbs sampler. Journal of the American Statistical Association, 87(419):pp. 861--868, 1992.
[9]
M. Rosen-Zvi, T. Griffiths, M. Steyvers, and P. Smyth. The author-topic model for authors and documents. UAI, 2004.
[10]
R. Socher, S. Gershman, A. Perotte, and P. Sederberg. A bayesian analysis of dynamics in free recall. NIPS, 2009.
[11]
K. Sohn and E. Xing. A hierarchical dirichlet process mixture model for haplotype reconstruction from multi-population data. Annals of Applied Statistics, 3(2):791--821, 2009.
[12]
Y. Teh, M. Jordan, M. Beal, and D. Blei. Hierarchical dirichlet processes. Journal of the American Statistical Association, Jan 2006.
[13]
Y. Teh, K. Kurihara, and M. Welling. Collapsed variational inference for HDP. NIPS, 20, 2008.
[14]
H. Wallach, D. Mimno, and A. McCallum. Rethinking LDA: Why Priors Matter. NIPS, 2009.
[15]
C. Wang and D. Blei. Decoupling sparsity and smoothness in the discrete hierarchical dirichlet process. NIPS, 2010.
[16]
X. Wang and E. Grimson. Spatial latent dirichlet allocation. NIPS, 2007.
[17]
J. Zhang, Y. Song, C. Zhang, and S. Liu. Evolutionary hierarchical dirichlet processes for multiple correlated time-varying corpora. KDD, 2010.

Cited By

View all
  • (2021)Topic Modeling Using Latent Dirichlet allocationACM Computing Surveys10.1145/346247854:7(1-35)Online publication date: 17-Sep-2021
  • (2021)End-to-End Framework for Imputation and State Discovery in Longitudinal Energy DataProceedings of the Twelfth ACM International Conference on Future Energy Systems10.1145/3447555.3466588(475-482)Online publication date: 22-Jun-2021
  • (2021)An Attention Hierarchical Topic ModelingPattern Recognition and Image Analysis10.1134/S105466182104029531:4(722-729)Online publication date: 27-Dec-2021
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CIKM '11: Proceedings of the 20th ACM international conference on Information and knowledge management
October 2011
2712 pages
ISBN:9781450307178
DOI:10.1145/2063576
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 October 2011

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Bayesian nonparametric models
  2. latent topic modeling

Qualifiers

  • Research-article

Conference

CIKM '11
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)7
  • Downloads (Last 6 weeks)0
Reflects downloads up to 03 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2021)Topic Modeling Using Latent Dirichlet allocationACM Computing Surveys10.1145/346247854:7(1-35)Online publication date: 17-Sep-2021
  • (2021)End-to-End Framework for Imputation and State Discovery in Longitudinal Energy DataProceedings of the Twelfth ACM International Conference on Future Energy Systems10.1145/3447555.3466588(475-482)Online publication date: 22-Jun-2021
  • (2021)An Attention Hierarchical Topic ModelingPattern Recognition and Image Analysis10.1134/S105466182104029531:4(722-729)Online publication date: 27-Dec-2021
  • (2019)Building a TIN-LDA Model for Mining Microblog Users’ InterestIEEE Access10.1109/ACCESS.2019.28979107(21795-21806)Online publication date: 2019
  • (2019)Theme Evolution Analysis of Public Security Events Based on Hierarchical Dirichlet ProcessFrontier Computing10.1007/978-981-13-3648-5_5(40-49)Online publication date: 19-May-2019
  • (2018)FinancialFlow: Visual analytics of financial news based on hierarchical dirichlet process2018 33rd Youth Academic Annual Conference of Chinese Association of Automation (YAC)10.1109/YAC.2018.8406403(375-380)Online publication date: May-2018
  • (2018)HDP-TUB Based Topic Mining Method for Chinese Micro-blogsNatural Language Processing and Chinese Computing10.1007/978-3-319-73618-1_75(856-865)Online publication date: 5-Jan-2018
  • (2016)Data clustering using side information dependent Chinese restaurant processesKnowledge and Information Systems10.1007/s10115-015-0834-747:2(463-488)Online publication date: 1-May-2016
  • (2016)Discovering hierarchical topic evolution in time-stamped documentsJournal of the Association for Information Science and Technology10.1002/asi.2343967:4(915-927)Online publication date: 1-Apr-2016
  • (2015)Hierarchical Dirichlet Process Mixture Model for Music Emotion RecognitionIEEE Transactions on Affective Computing10.1109/TAFFC.2015.24152126:3(261-271)Online publication date: 1-Jul-2015
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media