Ensemble Learning with Decision Tree for Remote Sensing Classification

Authors: Mahesh Pal

Abstract:

In recent years, a number of works proposing the combination of multiple classifiers to produce a single classification have been reported in remote sensing literature. The resulting classifier, referred to as an ensemble classifier, is generally found to be more accurate than any of the individual classifiers making up the ensemble. As accuracy is the primary concern, much of the research in the field of land cover classification is focused on improving classification accuracy. This study compares the performance of four ensemble approaches (boosting, bagging, DECORATE and random subspace) with a univariate decision tree as base classifier. Two training datasets, one without ant noise and other with 20 percent noise was used to judge the performance of different ensemble approaches. Results with noise free data set suggest an improvement of about 4% in classification accuracy with all ensemble approaches in comparison to the results provided by univariate decision tree classifier. Highest classification accuracy of 87.43% was achieved by boosted decision tree. A comparison of results with noisy data set suggests that bagging, DECORATE and random subspace approaches works well with this data whereas the performance of boosted decision tree degrades and a classification accuracy of 79.7% is achieved which is even lower than that is achieved (i.e. 80.02%) by using unboosted decision tree classifier.

Keywords: Ensemble learning, decision tree, remote sensingclassification.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1328942

References:

[1] G. Giacinto and F. Roli, Ensembles of neural networks for soft classification of remote sensing images. Proceedings of the European Symposium on Intelligent Techniques, European Network for Fuzzy Logic and Uncertainty Modelling in Information Technology, Bari, Italy, 166-170, 1997.
[2] J. A.,Benediktsson, J. R.,Sveinsson, O. K.,Ersoy and P. H., Swain, Parallel consensual neural networks. IEEE Trans. Neural Networks, 8, 1997, 54-65.
[3] F. Roli, G. Giacinto and G. Vernazza, Comparison and combination of statistical and neural networks algorithms for remote-sensing image classification. Neurocomputation in Remote Sensing Data Analysis, Austin, J., Kanellopoulos, I., Roli, F. and Wilkinson G. (Eds.), Berlin: Springer-Verlag, 117-124, 1997.
[4] L., Breiman, Bagging predictors, Machine Learning, 26, 1996, 123- 140.
[5] Y.Freund and R. Schapire, Experiments with a new boosting algorithm. Machine Learning: Proceedings of the Thirteenth International conference, 148-156, 1996.
[6] M. A. Friedl, C. E. Brodley, and A. H. Strahler, Maximizing land cover classification accuracies produced by decision tree at continental to global scales. IEEE Transactions on Geoscience and Remote Sensing. 37, 1999, 969-977.
[7] M. Pal and P. M. Mather, Decision tree classifiers and land use classification. Proceedings of the 27th Annual Conference of the Remote Sensing Society, 12-14 September, London, UK, 2001.
[8] G. J., Briem, J. A.,Benediktsson, and J. R., Sveinsson, Multiple Classifiers Applied to Multisource Remote Sensing Data IEEE Transactions on Geoscience and Remote Sensing, 40, 2002, 2291- 2299.
[9] P. Melville and R. Mooney, Constructing diverse classifier ensembles using artificial training examples. In Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, 505-510, 2003, Acapulco, Mexico, August.
[10] T.K.Ho, The Random Subspace Method for Constructing Decision Forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 1998, 832-844.
[11] L. Hansen and P. Salamon, Neural network ensembles, IEEE Transactions on Pattern recognition and Machine intelligence, 12, 1990, 993-1001.
[12] A. Krogh and J. Vedelsby, Neural networks ensembles, cross validation and active learning. In D.S. Touretzky, G. Tesauro, and T.K. Leen, editors, Advances in Neural Information Processing Systems, volume 7, pages 107-115, 1995, MIT Press, Cambridge, MA.
[13] L.,Breiman, J.H. Friedman, R.A.,Olshen and C.J.,Stone, Classification and Regression Trees, Wadsworth, Monterey, CA, 1984.
[14] S. K.Murthy, S. Kasif and S. Salzberg, A system for induction of oblique decision trees. Journal of Artificial Intelligence Research, 2, 1994, 1-32.
[15] I. Kononenko and J. S. Hong, Attribute selection for modelling. Future Generation Computer Systems, 13, 1997, 181-195.
[16] J. Mingers, An empirical comparison of selection measures for decision tree induction. Machine Learning, 3, 1989, 319-342.
[17] J. R. Quinlan, C4.5: Programs for Machine Learning. San Mateo: Morgan Kaufmann, San Francisco, 1993.