Methods › General › Asynchronous Data Parallel › Wavelet Distributed Training
Wavelet Distributed Training
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Wavelet is an asynchronous data parallel approach that interleaves waves of training tasks on the same group of GPUs, such that tasks belong to one wave can leverage on-device memory from tasks in another wave during their memory valley period, thus boost-up the training throughput. As shown in the Figure, Wavelet divides dataparallel training tasks into two waves, namely tick-wave and tock-wave. The task launching offset is achieved by delaying the launch time of tock-wave tasks for half of a whole forward-backward training cycle. Therefore, the tock-wave tasks can directly leverage GPU memory valley period of tick-wave tasks (e.g. 0.4s-0.6s in Figure 2(a)), since backward propagation of tick-wave tasks is compute-heavy but memory is often unused. Similarly, tick-wave tasks can leverage memory valley period of tock-wave tasks in the same way.
In Syntology name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.
Near matches the lookup lists, each a different name: Wavelet ESN (wESN); Distributed Sentinel; Wavelet Flow Matching; Wavelet Flow Matching; Latent Wavelet Diffusion; Speaking in Wavelet Domain; distributed prompt caching; learnable wavelet activation.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 0 · with a code link: 0 · where Syntology ran a sample: 0 (0 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology
The archive tags no paper with this method.
Tasks archive 2025-07-28
The archive attaches no task to a paper tagged with this method.
Usage over time archive 2025-07-28
No dated papers to chart.
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections