Lab 1.2: Teach It 1s and 2s First, Then Introduce 3 — Watching Weights Reorganize

Mon, 19 Oct 2026 00:00:00 +0530

Lab 1.2: Teach It 1s and 2s First, Then Introduce 3 — Watching Weights Reorganize

MNIST internals series. In Lab 1.1 we established what a weight row is: a learned photograph of a class, contrast-carved against its rivals, painted by the update rule $W_c \leftarrow W_c - \eta\,\tfrac1N\sum (\hat p_c - y_c)\,x^\top$ .

Now the experiment that motivated the series: start the model in an impoverished world — only 1s and 2s exist — then let it discover 3. This is continual learning in miniature, and it lets us watch, directly and quantitatively:

Continual-Learning on Sanketh's Blog

Lab 1.2: Teach It 1s and 2s First, Then Introduce 3 — Watching Weights Reorganize

Lab 1.2: Teach It 1s and 2s First, Then Introduce 3 — Watching Weights Reorganize