Connectionist Approach to Generic Text Summarization

Authors: Rajesh S.Prasad, U. V. Kulkarni, Jayashree.R.Prasad

Abstract:

As the enormous amount of on-line text grows on the World-Wide Web, the development of methods for automatically summarizing this text becomes more important. The primary goal of this research is to create an efficient tool that is able to summarize large documents automatically. We propose an Evolving connectionist System that is adaptive, incremental learning and knowledge representation system that evolves its structure and functionality. In this paper, we propose a novel approach for Part of Speech disambiguation using a recurrent neural network, a paradigm capable of dealing with sequential data. We observed that connectionist approach to text summarization has a natural way of learning grammatical structures through experience. Experimental results show that our approach achieves acceptable performance.

Keywords: Artificial Neural Networks (ANN); Computational Intelligence (CI); Connectionist Text Summarizer ECTS (ECTS); Evolving Connectionist systems; Evolving systems; Fuzzy systems (FS); Part of Speech (POS) disambiguation.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1056248

References:

[1] Amari, S., "A Theory of Adaptive Pattern Classifiers", IEEE Trans., Electron. Comput. 16:1143-1163.
[2] Amari, S. and Kasabov, N., "Brain-like Computing and Intelligent Information System, Springer Verlag, Singapore.
[3] Arbib M, "The Handbook of Brain Theory and Neural Networks", MIT Press, Cambridge, MA, (1995, 2002).
[4] Barak, A. Pearlmutter, "Gradient calculation for dynamic recurrent neural networks: a survey", IEEE Transactions on Neural Networks, 6(5), pp. 1212-1228, 1995.
[5] Elman, Jeffrey L., "Distributed representations, Simple recurrent networks, and grammatical Structure", Machine Learning, 7, pp. 195- 225.
[6] Freeman, W., "Neurodynamics", Springer, London, 2000.
[7] Garen Arevian, "Recurrent Neural Networks for Robust Real-World Text Classification", IEEE/WIC/ACM International Conference on Web Intelligence, 2007.
[8] Han, K.-H. And Kim, J.-H. "Quantum-inspired evolutionary algorithm for a class of Combinational optimization", IEEE Transactions. Evol. Comput., Vol. 6, No. 6, pp 580-593.
[9] Hebb, D., "The Organization of Behavior", Willey, New York.Haykin, S., "Neural Networks: A Comprehensive Foundation", 2nd edition, Prentice Hall, 1998.
[10] Hongyan Jing, "Sentence Reduction for Automatic Text Summarization", Department of Computer Science Columbia University New York, NY 10027, USA
[11] J. L. Elman, "Distributed representations, simple recurrent networks, and grammatical structure," Machine Learning, vol. 7, pp. 195-225, 1991.
[12] Khosrow Kaikhah, "Text Summarization Using Neural Networks", Faculty Publications-Comp. Science Texas State University,2004.
[13] Lei Yu1, Jia Ma1, Fuji Ren1, 2, Shingo Kuroiwa1, "Automatic Text Summarization Based on Lexical Chains and Structural Features", ACIS, Journal of Artificial Intelligence Research (1997), pp 574-578.
[14] Medsker, Larry R., "Hybrid Neural Network and Expert Systems", Kluwer Academic Publishers.
[15] Mayberry,M. R. and Miikkulainen R., "Lexical Disambiguation on Distributed Representations of Context Frequency", In Proceedings of the 16th Annual Conference of the Cognitive Science Society (COGSCI- 94, Atlanta, GA), pp. 601-606.
[16] Nikola Kasabov, "Evolving Connectionist Systems", Springer, second edition, 2007.
[17] Rosch, E. and Lloyd, "Cognition and Categorization", Lawrence Erlbaum, Hillssale, NJ, 1978.
[18] Skapura, David M., "Building Neural Networks", ACM Press, New York, pp.154-161.
[19] Smith, E.E. and Medin, D.L. "Categories and Concepts", Harward University Press, Cambridge, MA, 1981.
[20] Stoianov, I. P., Nerbonne, J. and Bouma, H., "Modelling the Phonotactic Structure of Natural Language Words with Simple Recurrent Networks", Computational Linguistics in Netherlands 1997.
[21] Tebelskis, J., "Speech Recognition using Neural Networks", CMU-CS- 95-142, Carnegie Mellon University, Pittsburgh, PA, May 1995.
[22] Wermter, S., Weber V., "SCREEN: Learning a Flat Syntactic and Semantic Spoken Language Analysis Using Artificial Neural Network", pp. 35-85.
[23] William M. Ramsey, Stephen P. Stich, David E. Rumelhart, "Philosophy and connectionist theory", Lawrence Erlbaum Associates, 1991,ISBN 0805805923, 9780805805925
[24] Yau-Hwang Kuo and Hsun-Hui Huang, "Automatic Extraction of Key Sentences via Word Sense Identification for Chinese Text Summarization", Journal of Advanced Computational Intelligence Vol.11 No.4, 2007 and Intelligent Informatics.
[25] Taylor, J.G., "The Race for Consciousness", MIT Press, Cambridge, MA., 1999.