Transferable representation learning for drug discovery

Drug discovery seeks to identify new candidate medications that can effectively treat human diseases with acceptable developability. Traditional computational and machine learning methods leverage handcrafted domain features for drug screening but suffer from poor transferability due to the vast chemical search space. In this thesis, we introduce transferable representation learning, a prominent approach within deep learning, to address different domain transferability challenges in drug discovery. Specifically, we develop three deep learning-based frameworks to learn transferable representations that adapt to key drug-related tasks, improving specific transferability for drug-target interaction prediction and enhancing generic transferability for molecular property prediction and inverse protein folding.

For drug-target interaction prediction, we first propose a low-bias evaluation strategy to effectively validate specific transferability. After that, we develop a bilinear attention network-based framework incorporating domain adaptation to improve performance under both in-domain and cross-domain settings. Furthermore, we design a molecular self-supervised pre-training framework aimed at improving generic transferability for molecular property prediction. The pre-trained model fully captures 2D topological and 3D geometric information of molecules, enabling fine-tuning for different downstream property prediction tasks. Finally, we design a mask prior-guided denoising diffusion framework that improves generic transferability for inverse protein folding, which involves iteratively generating feasible amino acid sequences that can fold into a given protein structure. In this thesis, extensive experiments are conducted to demonstrate the effectiveness of our proposed frameworks compared to related state-of-the-art methods. We also identify potential research directions in this emerging field for future exploration.