The microscope is the mask: privileged views and labels from a cryo-ET forward model
The researchers proposed a model named CARNIVAL, which uses simulated data to train protein annotation models to address the challenges caused by limited tilt angles and severely corrupted measurement operators in冷冻 electron tomography. The study first utilized the corrosion generated by the forward model to create domain-specific enhanced paired views, integrating them into the LeJEPA self-supervised training framework to achieve invariance goals; secondly, it optimized the model architecture and loss function using protein positions and identity information from simulated processes, enabling semantic information to be located at protein positions within dense feature volumes. Without fine-tuning, CARNIVAL was evaluated in classification and detection tasks on a benchmark dataset containing various protein types and two types of tomography processing. The results showed that CARNIVAL performed better than the current state-of-the-art models that were trained on simulated data using only contrast targets, without using forward model paired views or privileged information before training.