Modifying boosted trees to improve performance on task 1 of the 2006 KDD challenge cup

ABSTRACT

Task 1 of the 2006 KDD Challenge Cup required classification of pulmonary embolisms (PEs) using variables derived from computed tomography angiography. We present our approach to the challenge and justification for our choices. We used boosted trees to perform the main classification task, but modified the algorithm to address idiosyncrasies of the scoring criteria. The two main modifications were: 1) changing the dependent variable in the training set to account for multiple PEs per patient, and 2) incorporating neighborhood information through augmentation of the set of predictor variables. Both of these resulted in measurable predictive improvement. In addition, we discuss a statistically based method for setting the classification threshold.

REFERENCES

• 1: Breiman, L. Arcing classifiers (with discussion and rejoinder). Annals Statist. 26 (1998), 801--849
• 2: Friedman, J. H. Greedy function approximation: a gradient boosting machine, Annals Statist. 29 (2001), 1189--1232.
• 3: Friedman, J., Hastie, T., and Tibshirani, R. Additive logistic regression: a statistical view of boosting (with discussion and rejoinder). Annals Statist. 28 (2000), 337--407.
• 4: Hastie, T., Tibshirani, R., and Friedman, J. The Elements of Statistical Learning. Springer, New York, 2001.
• 5: KDD Cup 2006. Data description, task specification, rules. http://www.cs.unm.edu/files/kdd-cup-2006-task-spec-v1.pdf
• 6: Robert E. Schapire , Yoram Singer, Improved Boosting Algorithms Using Confidence-rated Predictions, Machine Learning, v.37 n.3, p.297-336, Dec. 1999 [doi>10.1023/A:1007614523901 ]
• 7: Viola, P., Platt, J, and Zhang, C. Multiple instance boosting for object detection. NIPS 18, 2006.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.1 Models
          Subjects: Statistical

Keywords:
boosted trees, classification, multiple instance learning, nonstandard loss function, real AdaBoost