Fractal and Chaotic Activation Functions in Echo State Networks: Preprocessing Topology Governs the Echo State Property

Through a parameter scan of 36,610 reservoirs, the researchers systematically studied the performance of non-smooth activation functions such as chaos, randomness, and fractals (e.g., Cantor) in echo-state networks. It was found that compared to traditional smooth activation functions (such as tanh and ReLU), these non-smooth functions not only maintain the echo-state property (ESP), but also perform better in terms of convergence speed and spectral radius tolerance; specifically, the Cantor function is 2.6 times faster than tanh and ReLU under certain parameters. The study proposed a theoretical framework for quantifying the degradation of the echo-state property by activation functions and demonstrated that this property implies traditional ESP. Additionally, a failure prediction model based on critical crowding ratio was established. Analysis showed that the preprocessing topology, rather than the continuity of the function, is the key factor determining network stability: monotonic compressive preprocessing can maintain ESP across scales, while divergent or discontinuous preprocessing leads to a sharp decline in performance.