An Analysis of Self-supervised Pre-training with Dependent Samples

SummaryAI generated

On September 7, 2026, a research paper titled “An Analysis of Self-supervised Pre-training with Dependent Samples” was published in the arXiv cs.LG domain. This study aims to analyze methods of self-supervised pre-training under the condition of dependent samples.

Related eventsRELATED EVENTS

2026-09-09 05:29OpenAI claims huge maths breakthrough on a famed ‘Millennium Problem’
2026-09-09 06:11OpenAI claims to have solved the Navier-Stokes equation problem, but its originality is questioned by the academic community
2026-09-08 22:00Google DeepMind Maps 9 Billion Possible DNA Variants

All reports (1)SOURCES

↗

A arXiv cs.LG en 2026-09-07 12:00

本文分析表明，在自监督预训练中，尽管不同增强样本间存在依赖关系，但将它们合并（pooling）比将其划分为独立子集处理是更优方案。具体而言，在估计不变子空间时，合并增强的统计误差上界从不劣于划分基线；在某些情况（如浅层神经网络上的掩码或噪声注入增强）下，朴素合并还能带来更快的收敛速率。该分析揭示了合并增强样本成功的内在机制，并解释了实践中倾向于使用大量增强的原因。