Discriminative learning for differing training and test distributions

ABSTRACT

We address classification problems for which the training instances are governed by a distribution that is allowed to differ arbitrarily from the test distribution---problems also referred to as classification under covariate shift. We derive a solution that is purely discriminative: neither training nor test distribution are modeled explicitly. We formulate the general problem of learning under covariate shift as an integrated optimization problem. We derive a kernel logistic regression classifier for differing training and test distributions.

REFERENCES

• 1: Bickel, S., & Scheffer, T. (2007). Dirichlet-enhanced spam filtering based on biased samples. Advances in Neural Information Processing Systems.
• 2: Dudik, M., Schapire, R., & Phillips, S. (2005). Correcting sample selection bias in maximum entropy density estimation. Advances in Neural Information Processing Systems.
• 3: Elkan, C. (2001). The foundations of cost-sensitive learning. Proceedings of the International Joint Conference on Artificial Intellligence.
• 4: Heckman, J. (1979). Sample selection bias as a specification error. Econometrica, 47, 153--161.
• 5: Huang, J., Smola, A., Gretton, A., Borgwardt, K., & Schöölkopf, B. (2007). Correcting sample selection bias by unlabeled data. Advances in Neural Information Processing Systems.
• 6: Japkowicz, N., & Stephen, S. (2002). The class imbalance problem: A systematic study. Intelligent Data Analysis, 6, 429--449.
• 7: Manski, C., & Lerman, S. (1977). The estimation of choice probabilities from choice based samples. Econometrica, 45, 1977--1988.
• 8: Shimodaira, H. (2000). Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of Statistical Planning and Inference, 90, 227--244.
• 9: Silverman, B. W. (1986). Density estimation for statistics and data analysis. Chapman & Hall, London.
• 10: Sugiyama, M., & Müüller, K.-R. (2005). Model selection under covariate shift. Proceedings of the International Conference on Artificial Neural Networks.
• 11: Ya Xue , Xuejun Liao , Lawrence Carin , Balaji Krishnapuram, Multi-Task Learning for Classification with Dirichlet Process Priors, The Journal of Machine Learning Research, 8, p.35-63, 5/1/2007
• 12: Bianca Zadrozny, Learning and evaluating classifiers under sample selection bias, Proceedings of the twenty-first international conference on Machine learning, p.114, July 04-08, 2004, Banff, Alberta, Canada [doi>10.1145/1015330.1015425]