ABSTRACT
Most current multi-task learning frameworks ignore the robustness issue, which means that the presence of "outlier" tasks may greatly reduce overall system performance. We introduce a robust framework for Bayesian multitask learning, t-processes (TP), which are a generalization of Gaussian processes (GP) for multi-task learning. TP allows the system to effectively distinguish good tasks from noisy or outlier tasks. Experiments show that TP not only improves overall system performance, but can also serve as an indicator for the "informativeness" of different tasks.
- Ando, R. K., & Zhang, T. (2005). A framework for learning predictive structures from multiple tasks and unlabeled data. JMLR, 6. Google ScholarDigital Library
- Argyriou, A., Evgeniou, T., & Pontil, M. (2006). Multitask feature learning. NIPS'06.Google Scholar
- Bakker, B., & Heskes, T. (2003). Task clustering and gating for Bayesian multitask learning. JMLR, 4, 83--89. Google ScholarDigital Library
- Caruana, R. (1997). Multitask learning. Machine Learning, 28, 41--75. Google ScholarDigital Library
- Evgeniou, T., & Pontil, M. (2004). Regularized multi-task learning. Proceedings SIGKDD. Google ScholarDigital Library
- Fernandez, C., & Steel, M. F. J. (1999). Multivariate Student-t Regression Models: Pitfalls and Inference. Biometrika, 86, 153--167.Google ScholarCross Ref
- Gelman, A., Carlin, J. B., Stern, H., & Rubin, D. B. (1996). Bayesian data analysis. Chapman and Hall-CRC.Google Scholar
- Jordan, M. I., Ghahramani, Z., Jaakkola, T., & Saul, L. K. (1999). An introduction to variational methods for graphical models. Machine Learning, 37, 183--233. Google ScholarDigital Library
- Kotz, S., & Nadarajah, S. (2004). Multivariate t-distributions and their applications. Cambridge University Press.Google Scholar
- Liu, C., & Rubin, D. B. (1995). ML Estimation of the t Distribution using EM and its Extensions, ECM and ECME. Statistica Sinica, 5, 19--39.Google Scholar
- Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian processes for machine learning. MIT Press. Google ScholarDigital Library
- Schölkopf, B., & Smola, A. J. (2002). Learning with kernels. MIT Press.Google Scholar
- Schwaighofer, A., Tresp, V., & Yu, K. (2005). Hierarchical bayesian modelling with Gaussian processes. NIPS'05.Google Scholar
- Yu, K., Tresp, V., & Schwaighofer, A. (2005). Learning Gaussian processes from multiple tasks. ICML'05. Google ScholarDigital Library
- Yu, S., Yu, K., Tresp, V., & Kriegel, H.-P. (2006). Collaborative ordinal regression. ICML'06. Google ScholarDigital Library
- Zhang, J., Ghahramani, Z., & Yang, Y. (2005). Learning multiple related tasks using latent independent component analysis. NIPS'05.Google Scholar
Robust multi-task learning with t-processes
Recommendations
Robust multi-task feature learning
KDD '12: Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data miningMulti-task learning (MTL) aims to improve the performance of multiple related tasks by exploiting the intrinsic relationships among them. Recently, multi-task feature learning algorithms have received increasing attention and they have been successfully ...
Focused multi-task learning using gaussian processes
ECML PKDD'11: Proceedings of the 2011 European conference on Machine learning and knowledge discovery in databases - Volume Part IIGiven a learning task for a data set, learning it together with related tasks (data sets) can improve performance. Gaussian process models have been applied to such multi-task learning scenarios, based on joint priors for functions underlying the tasks. ...
Focused multi-task learning using Gaussian processes
ECMLPKDD'11: Proceedings of the 2011th European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part IIGiven a learning task for a data set, learning it together with related tasks (data sets) can improve performance. Gaussian process models have been applied to such multi-task learning scenarios, based on joint priors for functions underlying the tasks. ...
Comments