Self-Supervised Pretraining of Molecular Graph Encoders with LeJEPA

A study on molecular graph neural networks evaluated the performance of the LeJEPA architecture in self-supervised pre-training. The experiment was based on the antibiotic activity dataset by Wong et al. and the ogbg-molhiv task, using D-MPNN encoders in GPS and Chemprop styles for substructure and bootstrap evaluations. Results showed that LeJEPA pre-training significantly improved the quality of learned molecular representations, with the ROC-AUC of probes frozen in the ogbg-molhiv task rising from 0.665 to 0.788. However, this pre-training did not significantly improve the overall fine-tuning performance of the model; it was statistically significant only in the antibiotic skeleton classification test, and had no effect in random classification and D-MPNN fine-tuning experiments. The study indicates that the information provided by pre-training can further enhance performance through feature-level combinations (such as combining with Morgan fingerprints).