Performance

Lab 1: CPU Internals This is a hands-on report for the lab in /home/sanketh/cpu-lab on your Linux x86 laptop. It ties each experiment back to the ideas in content/posts/performance-engineering/2026-10-04-cpu-microarchitecture.markdown. Important note about this laptop: the CPU is an Intel Core i5-8250U (15 W mobile part, base 1.6 GHz, max 3.4 GHz). Under sustained load it thermally throttles, so absolute nanoseconds vary between runs. The conclusions come from ratios and TMA buckets, not from chasing the exact ns. ...

Performance Engineering: CPU Microarchitecture for the Practitioner This note explains what a modern CPU actually does while your program runs, and how to read that behavior through hardware performance counters. The goal is not to memorize every unit in a specific core design, but to build a mental model that makes perf output meaningful. If the previous note asked “where does the time go,” this note answers “what is the CPU doing with the cycles it consumes.” ...

Performance Engineering: A Practical Starting Map This is the first note in a performance-engineering sequence. It is not a summary of an SRE handbook, a cloud-observability guide, or a treatise on dashboard design. The goal is practical skill at the level of a single machine: understand why a program is slow, decide what to change, and verify that the change worked. The contexts that matter here are things like a chess engine, the microbenchmark problems on HFT University, a hot loop in the Linux kernel, or any single-node program where you own the source and the hardware. The operating system is the boundary we press against; the hardware is the ground truth. ...

CUDA Profiling and Performance Measurement The notes up to this point covered what makes CUDA kernels fast in theory: coalescing, tiling, occupancy, and asynchronous execution. This note covers how to measure whether a kernel is actually fast and how to find the bottleneck when it is not. Profiling is what separates guessing from optimization. Without it, you can apply every best practice and still be slower than a simpler implementation. ...

CUDA Streams and Host/Device Overlap The previous note covered reduction, the first parallel algorithm on the GPU side. This note moves to the host-device boundary: how to keep the GPU busy while the CPU prepares the next batch of data, and how to overlap data movement with computation. So far, every kernel launch and every cudaMemcpy has been synchronous. The host waits, the GPU works, the host waits again. CUDA streams let you break that pattern. ...

CUDA Occupancy: Filling the SM to Hide Latency The previous note showed how to use shared memory to fix uncoalesced access patterns. This note covers occupancy: how many warps can live on an SM at the same time, and why that number determines whether the GPU can hide memory latency. Occupancy is the first CUDA performance topic that is not about memory access at all. It is about keeping the warp schedulers busy. ...

CUDA Shared Memory Tiling The previous note showed that coalesced memory access is the key to memory-bound kernel performance. But some algorithms naturally want uncoalesced access patterns. The fix is shared-memory tiling: load data into fast on-chip SRAM in a coalesced way, rearrange it there, then write it back to global memory in a coalesced way. This note explains the tiling pattern, why __syncthreads() is essential, the bank-conflict problem, and how to choose tile sizes. ...

CUDA Memory Coalescing: Why Access Patterns Matter The previous note showed how to write and launch a CUDA kernel. This note answers the first performance question: why does the same kernel sometimes run 5× or 10× slower just because of how threads read memory? The answer is memory coalescing. Because GPU memory is optimized for bandwidth over latency, the hardware rewards access patterns where threads in a warp read or write contiguous addresses together. If they do not, the GPU wastes bandwidth and cycles fetching data that most threads ignore. ...