https://sankethbk.github.io/blog/tags/parallel-algorithms/Recent content in Parallel-Algorithms on Sanketh's BlogHugo -- 0.166.0en-usWed, 02 Sep 2026 00:00:00 +0000https://sankethbk.github.io/blog/posts/gpu/2026-09-02-1-cuda-reduction/Wed, 02 Sep 2026 00:00:00 +0000https://sankethbk.github.io/blog/posts/gpu/2026-09-02-1-cuda-reduction/<h1 id="cuda-reduction-parallel-sum-max-and-other-tree-algorithms">CUDA Reduction: Parallel Sum, Max, and Other Tree Algorithms</h1> <p>The <a href="https://sankethbk.github.io/blog/gpu/2026-09-01-2-cuda-occupancy">previous note</a> covered occupancy. This note covers <strong>reduction</strong>: taking a large array and producing a single value (a sum, max, min, product, or any associative binary operation). Reduction is one of the most important parallel algorithms on a GPU, and it teaches several key CUDA ideas at once: shared memory, thread cooperation, warp-level execution, and multi-kernel launches.</p> <p>This note builds the algorithm step by step, from a naive atomic version to a tree-based shared-memory reduction.</p>