skip to main content
10.1145/1273496.1273635acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicmlConference Proceedingsconference-collections
Article

Robust multi-task learning with t-processes

Published:20 June 2007Publication History

ABSTRACT

Most current multi-task learning frameworks ignore the robustness issue, which means that the presence of "outlier" tasks may greatly reduce overall system performance. We introduce a robust framework for Bayesian multitask learning, t-processes (TP), which are a generalization of Gaussian processes (GP) for multi-task learning. TP allows the system to effectively distinguish good tasks from noisy or outlier tasks. Experiments show that TP not only improves overall system performance, but can also serve as an indicator for the "informativeness" of different tasks.

References

  1. Ando, R. K., & Zhang, T. (2005). A framework for learning predictive structures from multiple tasks and unlabeled data. JMLR, 6. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Argyriou, A., Evgeniou, T., & Pontil, M. (2006). Multitask feature learning. NIPS'06.Google ScholarGoogle Scholar
  3. Bakker, B., & Heskes, T. (2003). Task clustering and gating for Bayesian multitask learning. JMLR, 4, 83--89. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Caruana, R. (1997). Multitask learning. Machine Learning, 28, 41--75. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Evgeniou, T., & Pontil, M. (2004). Regularized multi-task learning. Proceedings SIGKDD. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Fernandez, C., & Steel, M. F. J. (1999). Multivariate Student-t Regression Models: Pitfalls and Inference. Biometrika, 86, 153--167.Google ScholarGoogle ScholarCross RefCross Ref
  7. Gelman, A., Carlin, J. B., Stern, H., & Rubin, D. B. (1996). Bayesian data analysis. Chapman and Hall-CRC.Google ScholarGoogle Scholar
  8. Jordan, M. I., Ghahramani, Z., Jaakkola, T., & Saul, L. K. (1999). An introduction to variational methods for graphical models. Machine Learning, 37, 183--233. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Kotz, S., & Nadarajah, S. (2004). Multivariate t-distributions and their applications. Cambridge University Press.Google ScholarGoogle Scholar
  10. Liu, C., & Rubin, D. B. (1995). ML Estimation of the t Distribution using EM and its Extensions, ECM and ECME. Statistica Sinica, 5, 19--39.Google ScholarGoogle Scholar
  11. Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian processes for machine learning. MIT Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Schölkopf, B., & Smola, A. J. (2002). Learning with kernels. MIT Press.Google ScholarGoogle Scholar
  13. Schwaighofer, A., Tresp, V., & Yu, K. (2005). Hierarchical bayesian modelling with Gaussian processes. NIPS'05.Google ScholarGoogle Scholar
  14. Yu, K., Tresp, V., & Schwaighofer, A. (2005). Learning Gaussian processes from multiple tasks. ICML'05. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Yu, S., Yu, K., Tresp, V., & Kriegel, H.-P. (2006). Collaborative ordinal regression. ICML'06. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Zhang, J., Ghahramani, Z., & Yang, Y. (2005). Learning multiple related tasks using latent independent component analysis. NIPS'05.Google ScholarGoogle Scholar
  1. Robust multi-task learning with t-processes

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Other conferences
      ICML '07: Proceedings of the 24th international conference on Machine learning
      June 2007
      1233 pages
      ISBN:9781595937933
      DOI:10.1145/1273496

      Copyright © 2007 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 20 June 2007

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • Article

      Acceptance Rates

      Overall Acceptance Rate140of548submissions,26%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader