Deepmind | pre training of molecular property prediction through noise removal

This article introduces the University of Oxford Sheheryar Zaidi and DeepMind The company's Michael Schaarschmidt Jointly published articles 《Pre-training via Denoising for Molecular Property Prediction》. from 3D There is limited data in the structure to predict the molecular properties , This poses a challenge to the generalization of neural networks . The author introduces a pre training technique , It uses equilibrium 3D Large data sets of molecular structures to learn meaningful representations for downstream tasks . Inspired by recent noise regularization , The author's pre training goal is based on de-noising . It depends on the recognized connection between de-noising automatic encoder and fractional matching , The authors also show that the goal corresponds to learning the molecular force field directly from the equilibrium structure —— It is generated by Gaussian mixture approximation of physical state distribution . Experiments show that , Using this pre training goal can greatly improve the performance of multiple benchmarks , In widely used QM9 The data set has reached the most advanced level . Last , The author analyzes the influence of different factors on pre training , And put forward some practical opinions .

Thesis link ：

https://arxiv.org/abs/2206.00133

The contributions of the article are summarized as follows ：

• A simple and effective method is studied , By means of 3D Denoising in structure space to pre train , The aim is to improve from such 3D Prediction of the properties of downstream molecules in the structure . The de-noising target is proved to be related to learning a specific force field .

• Experiments show that pre training by de-noising can greatly improve the size of , Performance on task datasets with multiple challenges that differ in task nature and molecular location . This proves that the structural denoising is successfully transferred to the prediction of molecular properties , Especially in the widely used QM9 Data sets 12 One of the goals 10 Each goal has created the most advanced performance . chart 1 Illustrates the QM9 Performance of one of the goals .

• Author's comments on common GNN The architecture has been improved , In particular, it shows how to make a custom activation Converter （TAT） Applied to graph network simulator （GNS）, This complements pre training and further improves performance .

• Learn more about training set size 、 Model size and structure 、 And the relationship between upstream and downstream data sets , The author analyzes the benefits of pre training .