Split-LLM training gradient leakage privacy: Cloud-based systems use zero-gradient mode to accurately reconstruct real data rows

Recent research on the Split-LLM training system has revealed serious privacy vulnerabilities. Although local nodes attempt to hide data features through techniques such as potential spatial bottlenecks, Gaussian noise, and row reordering, the returned gradient backpropagation attack successfully bypasses the protection mechanisms. Since the loss function ignores synthetic bait rows, resulting in zero gradients, attackers can accurately identify and reverse-engineer the real private data rows by detecting zero-gradient patterns. Experiments show that in nine seed experiments, all test samples were fully identified, and approximately one additional token could be recovered for every hundred tokens. Although truncating and noise-ing gradients theoretically could mitigate this problem, existing systems still do not meet security standards because they do not cover five potential attack scenarios, including cumulative observations across training steps.