https://sankethbk.github.io/blog/tags/overlap/Recent content in Overlap on Sanketh's BlogHugo -- 0.166.0en-usWed, 02 Sep 2026 00:00:00 +0000https://sankethbk.github.io/blog/posts/gpu/2026-09-02-2-cuda-streams/Wed, 02 Sep 2026 00:00:00 +0000https://sankethbk.github.io/blog/posts/gpu/2026-09-02-2-cuda-streams/<h1 id="cuda-streams-and-hostdevice-overlap">CUDA Streams and Host/Device Overlap</h1> <p>The <a href="https://sankethbk.github.io/blog/gpu/2026-09-02-1-cuda-reduction">previous note</a> covered reduction, the first parallel algorithm on the GPU side. This note moves to the <strong>host-device boundary</strong>: how to keep the GPU busy while the CPU prepares the next batch of data, and how to overlap data movement with computation.</p> <p>So far, every kernel launch and every <code>cudaMemcpy</code> has been synchronous. The host waits, the GPU works, the host waits again. CUDA <strong>streams</strong> let you break that pattern.</p>