Multimedia Data Fusion for Event Detection in Twitter by Using Dempster-Shafer Evidence Theory

Authors: Samar M. Alqhtani, Suhuai Luo, Brian Regan

Abstract:

Data fusion technology can be the best way to extract useful information from multiple sources of data. It has been widely applied in various applications. This paper presents a data fusion approach in multimedia data for event detection in twitter by using Dempster-Shafer evidence theory. The methodology applies a mining algorithm to detect the event. There are two types of data in the fusion. The first is features extracted from text by using the bag-ofwords method which is calculated using the term frequency-inverse document frequency (TF-IDF). The second is the visual features extracted by applying scale-invariant feature transform (SIFT). The Dempster - Shafer theory of evidence is applied in order to fuse the information from these two sources. Our experiments have indicated that comparing to the approaches using individual data source, the proposed data fusion approach can increase the prediction accuracy for event detection. The experimental result showed that the proposed method achieved a high accuracy of 0.97, comparing with 0.93 with texts only, and 0.86 with images only.

Keywords: Data fusion, Dempster-Shafer theory, data mining, event detection.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1338632

References:

[1] Z. Xie and L. Guan, "Multimodal information fusion of audiovisual emotion recognition using novel information theoretic tools," in IEEE International Conference on Multimedia and Expo (ICME), 2013, pp. 1- 6.
[2] S. Zhou, H. Leung, and F. Yao, "Multimedia Data Fusion," Mathematical Problems in Engineering, vol. 2013, 2013.
[3] L. A. Klein, Sensor and data fusion: a tool for information assessment and decision making vol. 324: SPIE press Bellingham^ eWA WA, 2004.
[4] Mark Maybury and S. Walter, "Multimedia Information Extraction," 2008.
[5] P. K. Atrey, M. A. Hossain, A. El Saddik, and M. S. Kankanhalli, "Multimodal fusion for multimedia analysis: a survey," Multimedia systems, vol. 16, pp. 345-379, 2010.
[6] P. Sharma and M. Kaur, "Multimodal classification using feature level fusion and SVM," Int. J. Comput. Appl, vol. 76, pp. 26-32, 2013.
[7] J.-P. Isson and J. Harriott, Win with advanced business analytics: creating business value from your data: John Wiley & Sons, 2012.
[8] N. Mishra and D. S. Silakari, "Image mining in the context of content based image retrieval: a perspective," IJCSI International Journal of Computer Science Issues, vol. 9, pp. 98-107, 2012.
[9] Y. Kompatsiaris and P. Hobson, Semantic Multimedia and Ontologies: Springer, 2008.
[10] A. Ma, A. Flenner, D. Needell, and A. G. Percus, "Improving Image Clustering using Sparse Text and the Wisdom of the Crowds," arXiv preprint arXiv:1405.2102, 2014.
[11] S. M. Alqhtani, S. Luo, and B. Regan, "Fusing Text and Image for Event Detection in Twitter," arXiv preprint arXiv:1503.03920, 2015.
[12] J. Mao, Multimodal Data Fusion As a Predictior of Missing Information in Social Networks: Arizona State University, 2012.
[13] F. Atefeh and W. Khreich, "A survey of techniques for event detection in Twitter," Computational Intelligence, vol. Volume 31, 2013.
[14] G. Shafer, "A Mathematical Theory of Evidence," AMC, vol. 10, p. 12, 1976.
[15] J. Li, S. Luo, and J. S. Jin, "Sensor data fusion for accurate cloud presence prediction using Dempster-Shafer evidence theory," Sensors, vol. 10, pp. 9384-9396, 2010.
[16] Z. Zhang, T. Liu, and W. Zhang, "Novel Paradigm for Constructing Masses in Dempster-Shafer Evidence Theory for Wireless Sensor Network’s Multisource Data Fusion," Sensors, vol. 14, pp. 7049-7065, 2014.
[17] J. G. Li, C. N. Zheng, H. W. Xuan, and Y. Jiang, "Data Fusion in Environment Monitoring Systems with Extended Dempster-Shafer Theory," Applied Mechanics and Materials, vol. 543, pp. 1074-1077, 2014.
[18] O. Ozdikis, H. Oguztuzun, and P. Karagoz, "Evidential location estimation for events detected in twitter," in Proceedings of the 7th Workshop on Geographic Information Retrieval, 2013, pp. 9-16.
[19] G. Salton and C. Buckley, "Term-weighting approaches in automatic text retrieval," Information processing & management, vol. 24, pp. 513- 523, 1988.
[20] S. Wold, K. Esbensen, and P. Geladi, "Principal component analysis," Chemometrics and intelligent laboratory systems, vol. 2, pp. 37-52, 1987.
[21] D. G. Lowe, "Distinctive image features from scale-invariant keypoints," International journal of computer vision, vol. 60, pp. 91-110, 2004.
[22] R. M. Gray, "Vector quantization," ASSP Magazine, IEEE, vol. 1, pp. 4- 29, 1984.