An Attentional Bi-Stream Sequence Learner for Credit Card Fraud Detection

Authors: Mohsen Hasirian, Amir Shahab Shahabi

Abstract:

Modern societies, marked by expansive Internet connectivity and the rise of e-commerce, are now integrated with digital platforms at an unprecedented level. The efficiency, speed, and accessibility of e-commerce have garnered a substantial consumer base. Against this backdrop, electronic banking has undergone rapid proliferation within the realm of online activities. However, this growth has inadvertently given rise to an environment conducive to illicit activities, notably electronic payment fraud, posing a formidable challenge to the domain of electronic banking. A pivotal role in upholding the integrity of electronic commerce and business transactions is played by electronic fraud detection, particularly in the context of credit cards which underscores the imperative of comprehensive research in this field. To this end, our study presents an Attentional Bi-Stream Sequence Learner (AttBiSeL) framework that leverages attention mechanism and recurrent networks. By incorporating bidirectional recurrent layers, specifically bidirectional Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) layers, the proposed model adeptly extracts past and future transaction sequences, while accounting for the temporal flow of information in both directions. Moreover, the integration of an attention mechanism accentuates specific transactions to varying degrees, as manifested in the output of the recurrent networks. The effectiveness of the proposed approach in automatic credit card fraud classification is evaluated on the European Cardholders' Fraud Dataset. Empirical results validate that the hybrid architectural paradigm presented in this study yields enhanced accuracy compared to previous studies.

Keywords: Attention mechanism, credit card fraud, deep learning, recurrent neural network.

References:

[1] C. Phua, V. Lee, K. Smith, and R. Gayler, “A Comprehensive Survey of Data Mining-based Fraud Detection Research,” Computers in Human Behavior, vol. 28, no. 3, pp. 1002–1013, Sep. 2010, doi: 10.1016/j.chb.2012.01.002.
[2] A. Abdallah, M. A. Maarof, and A. Zainal, “Fraud detection system: A survey,” Journal of Network and Computer Applications, vol. 68, pp. 90–113, Jun. 2016, doi: 10.1016/J.JNCA.2016.04.007.
[3] T. Fischer and C. Krauss, “Deep learning with long short-term memory networks for financial market predictions,” European Journal of Operational Research, vol. 270, no. 2, pp. 654–669, Oct. 2018, doi: 10.1016/J.EJOR.2017.11.054.
[4] Y. Ding, W. Kang, J. Feng, B. Peng, and A. Yang, “Credit card fraud detection based on improved Variational Autoencoder Generative Adversarial Network,” IEEE Access, 2023, doi: 10.1109/ACCESS.2023.3302339.
[5] J. Jurgovsky et al., “Sequence classification for credit-card fraud detection,” Expert Systems with Applications, vol. 100, pp. 234–245, Jun. 2018, doi: 10.1016/J.ESWA.2018.01.037.
[6] A. Roy, J. Sun, R. Mahoney, L. Alonzi, S. Adams, and P. Beling, “Deep learning detecting fraud in credit card transactions,” 2018 Systems and Information Engineering Design Symposium, SIEDS 2018, pp. 129–134, Jun. 2018, doi: 10.1109/SIEDS.2018.8374722.
[7] A. Vaswani et al., “Attention Is All You Need,” Advances in Neural Information Processing Systems, vol. 2017-December, pp. 5999–6009, Jun. 2017, doi: 10.48550/arxiv.1706.03762.
[8] H. M. Gomes, J. P. Barddal, A. F. Enembreck, and A. Bifet, “A Survey on Ensemble Learning for Data Stream Classification,” ACM Computing Surveys (CSUR), vol. 50, no. 2, Mar. 2017, doi: 10.1145/3054925.
[9] A. D. Pozzolo, O. Caelen, R. A. Johnson, and G. Bontempi, “Calibrating probability with undersampling for unbalanced classification,” Proceedings - 2015 IEEE Symposium Series on Computational Intelligence, SSCI 2015, pp. 159–166, 2015, doi: 10.1109/SSCI.2015.33.
[10] C. Sweetlin Hemalatha, V. Vaidehi, and R. Lakshmi, “Minimal infrequent pattern based approach for mining outliers in data streams,” Expert Systems with Applications, vol. 42, no. 4, pp. 1998–2012, Mar. 2015, doi: 10.1016/J.ESWA.2014.09.053.
[11] A. RB and S. K. KR, “Credit card fraud detection using artificial neural network,” Global Transitions Proceedings, vol. 2, no. 1, pp. 35–41, Jun. 2021, doi: 10.1016/J.GLTP.2021.01.006.
[12] A. Srivastava, A. Kundu, S. Sural, and A. K. Majumdar, “Credit card fraud detection using Hidden Markov Model,” IEEE Transactions on Dependable and Secure Computing, vol. 5, no. 1, pp. 37–48, 2008, doi: 10.1109/TDSC.2007.70228.
[13] A. Dal Pozzolo, R. Johnson, O. Caelen, S. Waterschoot, N. V. Chawla, and G. Bontempi, “Using HDDT to avoid instances propagation in unbalanced and evolving data streams,” Proceedings of the International Joint Conference on Neural Networks, pp. 588–594, Sep. 2014, doi: 10.1109/IJCNN.2014.6889638.
[14] M. S. Kumar, V. Soundarya, S. Kavitha, E. S. Keerthika, and E. Aswini, “Credit Card Fraud Detection Using Random Forest Algorithm,” 2019 Proceedings of the 3rd International Conference on Computing and Communications Technologies, ICCCT 2019, pp. 149–153, Feb. 2019, doi: 10.1109/ICCCT2.2019.8824930.
[15] S. Bagga, A. Goyal, N. Gupta, and A. Goyal, “Credit Card Fraud Detection using Pipeling and Ensemble Learning,” Procedia Computer Science, vol. 173, pp. 104–112, Jan. 2020, doi: 10.1016/J.PROCS.2020.06.014.
[16] Y. F. Zhang, H. L. Lu, H. F. Lin, X. C. Qiao, and H. Zheng, “The Optimized Anomaly Detection Models Based on an Approach of Dealing with Imbalanced Dataset for Credit Card Fraud Detection,” Mobile Information Systems, vol. 2022, no. 1, p. 8027903, Jan. 2022, doi: 10.1155/2022/8027903.
[17] S. Xuan, G. Liu, Z. Li, L. Zheng, S. Wang, and C. Jiang, “Random forest for credit card fraud detection,” ICNSC 2018 - 15th IEEE International Conference on Networking, Sensing and Control, pp. 1–6, May 2018, doi: 10.1109/ICNSC.2018.8361343.
[18] S. Makki, Z. Assaghir, Y. Taher, R. Haque, M. S. Hacid, and H. Zeineddine, “An Experimental Study with Imbalanced Classification Approaches for Credit Card Fraud Detection,” IEEE Access, vol. 7, pp. 93010–93022, 2019, doi: 10.1109/ACCESS.2019.2927266.
[19] C. Jiang, J. Song, G. Liu, L. Zheng, and W. Luan, “Credit Card Fraud Detection: A Novel Approach Using Aggregation Strategy and Feedback Mechanism,” IEEE Internet of Things Journal, vol. 5, no. 5, pp. 3637–3647, Oct. 2018, doi: 10.1109/JIOT.2018.2816007.
[20] I. Benchaji, S. Douzi, and B. El Ouahidi, “Novel learning strategy based on genetic programming for credit card fraud detection in big data,” Multi Conference on Computer Science and Information Systems, MCCSIS 2019 - Proceedings of the International Conferences on Big Data Analytics, Data Mining and Computational Intelligence 2019 and Theory and Practice in Modern Computing 2019, pp. 3–10, 2019, doi: 10.33965/BIGDACI2019_201907L001.
[21] N. Mahmoudi and E. Duman, “Detecting credit card fraud by Modified Fisher Discriminant Analysis,” Expert Systems with Applications, vol. 42, no. 5, pp. 2510–2516, Apr. 2015, doi: 10.1016/J.ESWA.2014.10.037.
[22] A. Dal Pozzolo, G. Boracchi, O. Caelen, C. Alippi, and G. Bontempi, “Credit card fraud detection: A realistic modeling and a novel learning strategy,” IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 8, pp. 3784–3797, Aug. 2018, doi: 10.1109/TNNLS.2017.2736643.
[23] M. Z. Mizher and A. B. Nassif, “Deep CNN approach for Unbalanced Credit Card Fraud Detection Data,” 2023 Advances in Science and Engineering Technology International Conferences, ASET 2023, 2023, doi: 10.1109/ASET56582.2023.10180615.
[24] J. Forough and S. Momtazi, “Ensemble of deep sequential models for credit card fraud detection,” Applied Soft Computing, vol. 99, p. 106883, Feb. 2021, doi: 10.1016/J.ASOC.2020.106883.
[25] D. Bahdanau, K. H. Cho, and Y. Bengio, “Neural Machine Translation by Jointly Learning to Align and Translate,” 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, Sep. 2014, Accessed: May 21, 2023. Online. Available: https://arxiv.org/abs/1409.0473v7
[26] K. Xu et al., “Show, Attend and Tell: Neural Image Caption Generation with Visual Attention,” 32nd International Conference on Machine Learning, ICML 2015, vol. 3, pp. 2048–2057, Feb. 2015, Accessed: May 21, 2023. Online. Available: https://arxiv.org/abs/1502.03044v3
[27] Z. Yang, D. Yang, C. Dyer, X. He, A. Smola, and E. Hovy, “Hierarchical Attention Networks for Document Classification,” 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference, pp. 1480–1489, 2016, doi: 10.18653/V1/N16-1174.
[28] B. Lebichot, Y.-A. Le Borgne, L. He-Guelton, F. Oblé, and G. Bontempi, “Deep-Learning Domain Adaptation Techniques for Credit Cards Fraud Detection,” in INNS Big Data and Deep Learning conference, 2020, pp. 78–88. doi: 10.1007/978-3-030-16841-4_8.
[29] K. Fu, D. Cheng, Y. Tu, and L. Zhang, “Credit Card Fraud Detection Using Convolutional Neural Networks,” pp. 483–490, 2016, doi: 10.1007/978-3-319-46675-0_53.
[30] A. Somasundaram and S. Reddy, “Parallel and incremental credit card fraud detection model to handle concept drift and data imbalance,” Neural Computing and Applications, vol. 31, no. 1, pp. 3–14, Jan. 2019, doi: 10.1007/S00521-018-3633-8/METRICS.
[31] C. Cheadle, M. P. Vawter, W. J. Freed, and K. G. Becker, “Analysis of Microarray Data Using Z Score Transformation,” The Journal of Molecular Diagnostics, vol. 5, no. 2, pp. 73–81, May 2003, doi: 10.1016/S1525-1578(10)60455-2.
[32] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic minority over-sampling technique,” Journal of Artificial Intelligence Research, vol. 16, pp. 321–357, 2002, doi: 10.1613/jair.953.
[33] G. Liu and J. Guo, “Bidirectional LSTM with attention mechanism and convolutional layer for text classification,” Neurocomputing, vol. 337, pp. 325–338, Apr. 2019, doi: 10.1016/J.NEUCOM.2019.01.078.
[34] T. Chen, R. Xu, Y. He, Y. Xia, and X. Wang, “Learning User and Product Distributed Representations Using a Sequence Model for Sentiment Analysis,” IEEE Computational Intelligence Magazine, vol. 11, no. 3, pp. 34–44, Aug. 2016, doi: 10.1109/MCI.2016.2572539.
[35] J. K. Chorowski, D. Bahdanau, D. Serdyuk, K. Cho, and Y. Bengio, “Attention-Based Models for Speech Recognition,” Advances in Neural Information Processing Systems, vol. 28, 2015.
[36] H. G. K. A. S. I. S. R. S Nitish, “Dropout: a simple way to prevent neural networks from overfitting,” J Mach Learn Res., vol. 15, no. 1, pp. 1929–1958, 2014.
[37] M. Abadi et al., “Tensorflow: Large-scale machine learning on heterogeneous distributed systems,” arxiv.org, Online. Available: https://arxiv.org/abs/1603.04467
[38] D. P. Kingma and J. L. Ba, “Adam: A Method for Stochastic Optimization,” 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, Dec. 2014, Accessed: Jan. 09, 2022. Online. Available: https://arxiv.org/abs/1412.6980v9
[39] P. Vuttipittayamongkol, E. Elyan, and A. Petrovski, “On the class overlap problem in imbalanced data classification,” Knowledge-based systems, vol. 212, p. 106631, 2021.