BiFormerDTA: Structural Embedding of Protein in Drug Target Affinity Prediction Using BiFormer

Authors: Leila Baghaarabani, Parvin Razzaghi, Mennatolla Magdy Mostafa, Ahmad Albaqsami, Al Warith Al Rushaidi, Masoud Al Rawahi

Abstract:

Predicting the interaction between drugs and their molecular targets is pivotal for advancing drug development processes. Given the time and cost constraints, computational approaches have emerged as an effective approach to drug-target interaction (DTI) prediction. While most existing computational methods use drug molecules and protein sequences as inputs, this study goes further by introducing a protein representation developed using a masked protein language model. In this representation, each amino acid residue in the protein sequence is assigned a probability distribution, reflecting the likelihood of that residue occupying a specific position. The similarity between amino acid pairs is then calculated to generate a similarity matrix. To leverage this matrix, the study employs Bi-Level Routing Attention (BiFormer), a model that integrates transformer-based architectures with protein sequence analysis, representing a significant advancement in DTI prediction. BiFormer identifies the most critical regions of the protein sequence responsible for interactions with drugs, thereby deepening our understanding of these interactions. This approach demonstrates its ability to capture the local structural relationships within proteins and enhance the accuracy of DTI predictions. The proposed method was evaluated on two widely recognized datasets, Davis and KIBA, through comprehensive experiments that showcased its effectiveness compared to state-of-the-art techniques.

Keywords: BiFormer, transformer, protein language processing, self-attention mechanism, binding affinity, drug target interaction, similarity matrix, protein masked representation, protein language model.

References:

[1] John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool et al. Highly accurate protein structure prediction with AlphaFold Nature, 596(7873):583-589, 2021.
[2] Lei Zhu, Xinjiang Wang, Zhanghan Ke, Wayne Zhang, and Rynson Lau. BiFormer: Vision Transformer with Bi-Level Routing Attention. IEEE/CVF conference on computer vision and pattern recognition, 10323-10333, 2023.
[3] Qichang Zhao, Guihua Duan, Mengyun Yang, Zhongjian Cheng, Yaohang Li, and Jianxin Wang. AttentionDTA: Drug–Target Binding Affinity Prediction by Sequence-Based Deep Learning With Attention Mechanism. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 20(2):852–863, 2023.
[4] Mogan Gim, Junseok Choe, Seungheun Baek, Jueon Park, Chaeeun Lee, Minjae Ju, Sumin Lee, and Jaewoo Kang. ArkDTA: Attention Regularization Guided by Non-Covalent Interactions for Explainable Drug–Target Binding Affinity Prediction. Bioinformatics, 39(1):i448-i457, 2023.
[5] Xiangfeng Yan and Yong Liu. Graph-sequence attention and transformer for predicting drug-target affinity. RSC advances, 12(45):29525-29534, 2022.
[6] Haelee Bae and Hojung Nam. GraphATT-DTA: Attention-Based Novel Representation of Interaction to Predict Drug-Target Binding Affinity. Biomedicines, 11(1):67, 2022.
[7] Li Zhang, Chun-Chun Wang, Xing Chen. Predicting drug-target binding affinity through molecule representation block based on multi-head attention and skip connection. Briefings in Bioinformatics, 23(6):bbac468, 2022.
[8] Xiangfeng Yan, and Yong Liu. Graph–sequence attention and transformer for predicting drug–target affinity, RSC advances, 12(45):29525-29534, 2022.
[9] Mathijs Boezer, Maryam Tavakol, and Zahra Sajadi. FastDTI: drug-target interaction prediction using multimodality and transformers. Proceedings of the northern lights deep learning workshop, 4, 2023.
[10] Jiaxiao Chen, Zhonghui Gu, Youjun Xu, Minghua Deng, Luhua Lai, and Jianfeng Pei QuoteTarget: A sequence-based transformer protein language model to identify potentially druggable protein targets. Protein Science, 32(2):e4555, 2023.
[11] Junjie Wang, Jie Hu, Huiting Sun, MengDie Xu, Yun Yu, Yun Liu, and Liang Cheng. MGPLI: Exploring Multigranular Representations for Protein-Ligand Interaction Prediction. Bioinformatics, 38(21):4859-4867, 2022.
[12] Le Zhuo, Zewen Chi, Minghao Xu, Heyan Huang, Heqi Zheng, Conghui He, Xian-Ling Mao, Wentao Zhang. Protllm: An interleaved protein-language llm with protein-as-word pre-training. arXiv preprint arXiv:2403.07920, 2024.
[13] Ashish Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, volume 30, 2017.
[14] Jianhua Lin. Divergence Measures Based on the Shannon Entropy. IEEE Transactions on Information Theory, 37(1):145–151, 1991.
[15] Mindy I. Davis, Jeremy P. Hunt, Sanna Herrgard, Pietro Ciceri, Lisa M. Wodicka, Gabriel Pallares, Michael Hocker, Daniel K. Treiber, Patrick P. Zarrinkar. Comprehensive analysis of kinase inhibitor selectivity. Nature Biotechnology, 29(11):1046-1051, 2011.
[16] Jing Tang, Agnieszka Szwajda, Sushil Shakyawar, Tao Xu, Petteri Hintsanen, Krister Wennerberg, and Tero Aittokallio. Making sense of large-scale kinase inhibitor bioactivity data sets: a comparative and integrative analysis. Journal of Chemical Information and Modeling, 54(3):735-743, 2014.
[17] Shourun Pan, Leiming Xia, Lei Xu, and Zhen Li. SubMDTA: Drug Target Affinity Prediction Based on Substructure Extraction and Multi-scale Features. BMC Bioinformatics, 24(1):334, 2023.