CUDA Thread Hierarchy: Grids, Blocks, Warps, and Threads The previous note mapped the hardware: SM, warp scheduler, register file, shared memory, L2, device memory. This note maps the software abstraction CUDA exposes on top of that hardware. The two maps fit together almost one-to-one, and once you see how, most of CUDA stops being arbitrary syntax and becomes named hardware concepts. CUDA organizes parallel work into four nested levels: Grid └── Block (threads in a block can share fast on-chip memory, and sync) └── Warp (32 threads, executed in lockstep on the hardware) └── Thread (your kernel code, from one thread's point of view) This note works from the bottom up: start with the thread (the thing you actually program), then warp, then block, then grid, then show how the whole tower maps onto the GPU die. ...

Anatomy of a GPU: Hardware Components The previous note covered why GPUs look the way they do — throughput over latency, thousands of simple cores instead of a few smart ones. This note zooms into the actual silicon: what physically sits on a GPU die, what each piece is called, and what job it does. The goal is to have concrete hardware nouns (SM, warp scheduler, register file, L2, …) in hand before those same nouns start showing up as CUDA concepts (threadIdx, __shared__, occupancy, …). ...

CPU vs GPU Architecture Why GPUs exist For decades, single-thread CPU performance improved “for free” — you write the same sequential code, and it runs faster on the next generation of chips, because clock speeds kept climbing. Around the mid-2000s, that stopped. Clock speeds hit a power wall — you can’t keep cranking frequency without the chip melting. So the industry pivoted from “make one core faster” to “put more cores on the chip.” This is the multicore/manycore shift, and it’s the reason parallel programming stopped being a niche HPC skill and became something every programmer eventually runs into. ...

Page Frame Reclaiming The Page Frame Reclaiming Algorithm 1. Unified Page Cache & Dual-Caching Edge Case Unified Page Cache: Linux no longer separates the “disk/buffer cache” from the “page cache.” All file and block I/O goes through a single unified Page Cache indexed by (address_space, offset). The Dual-Caching Exception: Reading via a regular path (e.g., /app/data.db) uses the file inode’s address_space. Reading via the raw disk node (e.g., /dev/sda1) uses the block device inode’s address_space. ...

RISC-V (pronounced “risk-five”) is an open, royalty-free RISC ISA originally designed to support academic research and education, and now increasingly used in commercial silicon. Unlike MIPS, ARM, or x86, RISC-V is not owned by any single company — its specification is maintained by RISC-V International, and anyone is free to implement it without paying licensing fees. This openness, combined with a deliberately small and modular base ISA, has made it popular in everything from microcontrollers to research CPUs to (increasingly) high-performance application processors. ...

Pipelining — First Principles What is pipelining? Pipelining is the technique of overlapping the execution of multiple instructions. An instruction consists of multiple pieces of work. Instead of completing one instruction entirely before starting the next, we divide the work into stages and let different instructions occupy different stages simultaneously. For a simple example: IF ID EX MEM WB Instruction 1 → → → → → Instruction 2 → → → → Instruction 3 → → → Instruction 4 → → At any moment, several instructions are being worked on, but each is at a different stage. ...

Setuid Binaries # ls -la /bin/su -rwsr-xr-x 1 root root 67816 Aug 1 20:00 /bin/su Types of permissions and actions 1. The Three Types of Users (Who) Linux divides the entire universe of users into three distinct categories. Permissions are always listed in this exact order: Owner (User): The specific person who owns the file (usually the creator). Group: A defined collection of users who share access (e.g., a “developers” or “finance” group). Others: Everyone else on the system who is not the owner and not in the group. The owner and group are stored in inode. ...

Kernel Lab 2: Observing the Page Cache The Program The below program opens a file via mmap and performs read/write operations via commands. It also shows page faults, resident pages via mincore, etc. // page_cache_lab.c #define _GNU_SOURCE #include <errno.h> #include <fcntl.h> #include <inttypes.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <sys/mman.h> #include <sys/stat.h> #include <unistd.h> #include <sys/resource.h> static void print_help(void) { printf("\nCommands:\n"); printf(" help Show commands\n"); printf(" info Mapping information\n"); printf(" read <page> Read first byte of page\n"); printf(" write <page> <char> Fill entire page with character\n"); printf(" dump <page> Dump first 64 bytes\n"); printf(" msync Flush dirty pages\n"); printf(" dontneed madvise(MADV_DONTNEED)\n"); printf(" pause Wait for ENTER\n"); printf(" mincore Show all pages compact (R=resident .=not)\n"); printf(" mincore <page> Check if specific page is resident\n"); printf(" quit\n\n"); } // compact: prints [R....R..R] style, 64 pages per line static void print_mincore_compact(char *region, size_t filesize, long pagesize) { size_t npages = (filesize + pagesize - 1) / pagesize; unsigned char *vec = calloc(npages, 1); if (!vec) { perror("calloc"); return; } if (mincore(region, filesize, vec) != 0) { perror("mincore"); free(vec); return; } printf("\nPage residency (%zu pages, R=resident .=not):\n\n", npages); for (size_t i = 0; i < npages; i++) { if (i % 64 == 0) printf("%5zu: ", i); putchar((vec[i] & 1) ? 'R' : '.'); if ((i + 1) % 64 == 0 || i == npages - 1) putchar('\n'); } // print resident page numbers explicitly printf("\nResident pages: "); int any = 0; for (size_t i = 0; i < npages; i++) { if (vec[i] & 1) { printf("R(%zu) ", i); any = 1; } } if (!any) printf("none"); printf("\n\n"); free(vec); } // single page check static void print_mincore_page(char *region, size_t filesize, long pagesize, size_t page) { size_t npages = (filesize + pagesize - 1) / pagesize; if (page >= npages) { printf("Page %zu out of range (max %zu)\n", page, npages - 1); return; } unsigned char *vec = calloc(npages, 1); if (!vec) { perror("calloc"); return; } if (mincore(region, filesize, vec) != 0) { perror("mincore"); free(vec); return; } printf("Page %zu: %s\n", page, (vec[page] & 1) ? "resident" : "not resident"); free(vec); } typedef struct { unsigned long minflt; unsigned long majflt; char vmrss[64]; char rssanon[64]; char rssfile[64]; char rssshmem[64]; } ProcStats; static void get_proc_stats(ProcStats *s) { memset(s, 0, sizeof(*s)); /* ---------- /proc/self/stat ---------- */ FILE *fp = fopen("/proc/self/stat", "r"); if (fp) { char buf[4096]; if (fgets(buf, sizeof(buf), fp)) { /* Skip "pid (comm)" because comm may contain spaces */ char *p = strrchr(buf, ')'); if (p) { unsigned long values[64] = {0}; int n = 0; char *tok = strtok(p + 2, " "); while (tok && n < 64) { values[n++] = strtoul(tok, NULL, 10); tok = strtok(NULL, " "); } /* * After ')' the fields begin with: * * 0 state * 1 ppid * ... * 7 flags * 8 minflt * 10 majflt */ if (n > 10) { s->minflt = values[7]; s->majflt = values[9]; } } } fclose(fp); } /* ---------- /proc/self/status ---------- */ fp = fopen("/proc/self/status", "r"); if (fp) { char line[256]; while (fgets(line, sizeof(line), fp)) { sscanf(line, "VmRSS: %63[^\n]", s->vmrss); sscanf(line, "RssAnon: %63[^\n]", s->rssanon); sscanf(line, "RssFile: %63[^\n]", s->rssfile); sscanf(line, "RssShmem: %63[^\n]", s->rssshmem); } fclose(fp); } } static void print_stats(void) { static unsigned long last_min = 0; static unsigned long last_maj = 0; ProcStats s; get_proc_stats(&s); printf("\n---------------------------------------\n"); printf("Minor Faults : %-8lu (%+ld)\n", s.minflt, (long)s.minflt - (long)last_min); printf("Major Faults : %-8lu (%+ld)\n", s.majflt, (long)s.majflt - (long)last_maj); printf("---------------------------------------\n\n"); last_min = s.minflt; last_maj = s.majflt; } struct rusage ru; static long last_minflt = 0; static long last_majflt = 0; int main(int argc, char **argv) { if (argc != 2) { fprintf(stderr, "Usage: %s <file>\n", argv[0]); return 1; } const char *filename = argv[1]; int fd = open(filename, O_RDWR); if (fd < 0) { perror("open"); return 1; } struct stat st; if (fstat(fd, &st) != 0) { perror("fstat"); return 1; } size_t filesize = st.st_size; long pagesize = sysconf(_SC_PAGESIZE); if (filesize == 0) { fprintf(stderr, "File is empty.\n"); return 1; } char *region = mmap(NULL, filesize, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); if (region == MAP_FAILED) { perror("mmap"); return 1; } printf("\n=====================================\n"); printf("Page Cache Lab\n"); printf("=====================================\n"); printf("PID : %d\n", getpid()); printf("File : %s\n", filename); printf("Size : %zu bytes\n", filesize); printf("Page Size : %ld\n", pagesize); printf("Pages : %zu\n", filesize / pagesize); printf("Mapping : %p\n", region); print_help(); char line[256]; while (1) { printf("pagecache> "); fflush(stdout); if (!fgets(line, sizeof(line), stdin)) break; if (strncmp(line, "help", 4) == 0) { print_help(); } else if (strncmp(line, "info", 4) == 0) { printf("\n"); printf("PID : %d\n", getpid()); printf("Mapping : %p\n", region); printf("Size : %zu bytes\n", filesize); printf("Pages : %zu\n", filesize / pagesize); printf("\n"); } else if (strncmp(line, "read", 4) == 0) { size_t page; if (sscanf(line, "read %zu", &page) != 1) { printf("Usage: read <page>\n"); continue; } size_t offset = page * pagesize; if (offset >= filesize) { printf("Out of range\n"); continue; } volatile char c = region[offset]; printf("Read page %zu : '%c' (0x%02x)\n", page, (c >= 32 && c <= 126) ? c : '.', (unsigned char)c); } else if (strncmp(line, "write", 5) == 0) { size_t page; char ch; if (sscanf(line, "write %zu %c", &page, &ch) != 2) { printf("Usage: write <page> <char>\n"); continue; } size_t offset = page * pagesize; if (offset >= filesize) { printf("Out of range\n"); continue; } size_t remaining = filesize - offset; size_t len = remaining < (size_t)pagesize ? remaining : (size_t)pagesize; memset(region + offset, ch, len); printf("Filled page %zu with '%c'\n", page, ch); } else if (strncmp(line, "dump", 4) == 0) { size_t page; if (sscanf(line, "dump %zu", &page) != 1) { printf("Usage: dump <page>\n"); continue; } size_t offset = page * pagesize; if (offset >= filesize) { printf("Out of range\n"); continue; } printf("\n"); for (int i = 0; i < 64; i++) { if (offset + i >= filesize) break; unsigned char c = region[offset + i]; if (c >= 32 && c <= 126) putchar(c); else putchar('.'); } printf("\n\n"); } else if (strncmp(line, "msync", 5) == 0) { if (msync(region, filesize, MS_SYNC) != 0) perror("msync"); else printf("Pages flushed.\n"); } else if (strncmp(line, "dontneed", 8) == 0) { if (madvise(region, filesize, MADV_DONTNEED) != 0) perror("madvise"); else printf("MADV_DONTNEED completed.\n"); } else if (strncmp(line, "stats", 5) == 0) { print_stats(); continue; } else if (strncmp(line, "mincore", 7) == 0) { size_t page; if (sscanf(line, "mincore %zu", &page) == 1) { // single page query print_mincore_page(region, filesize, pagesize, page); } else { // compact full view print_mincore_compact(region, filesize, pagesize); } continue; } else if (strncmp(line, "pause", 5) == 0) { printf("Press ENTER..."); getchar(); } else if (strncmp(line, "quit", 4) == 0) { break; } else { printf("Unknown command.\n"); } print_stats(); } munmap(region, filesize); close(fd); return 0; } For the setup, i had to make some changes for this to work. ...

Kernel Lab 1: Observing Process and VFS Structs Primer on Kernel Data Structures 1. Process & Memory Management (MM) Struct Description Noteworthy Fields task_struct The kernel’s representation of a thread/process. mm (memory space), files (open files), fs (cwd/root), pid, comm (name). mm_struct The descriptor for an entire memory address space. mm_mt (the Maple Tree of VMAs), pgd (page global directory for hardware MMU). vm_area_struct A contiguous range of virtual memory with shared permissions (a VMA). vm_start, vm_end, vm_file (if it’s a memory-mapped file), vm_flags. 2. Virtual File System (VFS) The VFS bridges the gap between user-space file operations (like open() and read()) and the underlying hardware. ...

The Pipes File Descriptors To understand file descriptors (FDs), you have to look at one of the core design philosophies of Unix and Linux: “Everything is a file.” Whether a process is interacting with a regular text file, a directory, a hardware device (like a hard drive), a network socket, or an in-memory pipe, the kernel represents all of them as “files.” A file descriptor is simply the lightweight handle that user-space processes use to talk to these files. Here is the low-level architectural breakdown of how they work. ...