Interpretability

Lab 1.3: From Templates to Features — What One Hidden Layer Does to the Weights MNIST internals series. Lab 1.1 gave us the clean answer: in a linear softmax model, a weight row is a matched-filter photograph of its class. Lab 1.2 showed those photographs being painted, re-carved, and out-bid. Now the question this series was really building toward: add one hidden layer. What happens to the weight picture? If you’ve watched 3Blue1Brown’s neural nets series, you might expect: “layers learn edges, then curves, then parts; visualizing raw weights won’t tell you much here.” We can now test that ourselves instead of taking it on faith, and the truth lands in a more interesting place: ...

Lab 2.1: How Word Vectors Learn to Mean — Embeddings From Zero, and What They Secretly Compute Language-model internals series. In the MNIST series we took apart what weights represent for vision. New series, same ritual — build it from scratch, check gradients numerically, claim nothing without a baseline or a measured number. The question for this lab, in the plainest possible form: A word becomes a row of numbers. How can those numbers possibly mean anything? ...

Lab 1.1: What Do the Weights of a Neural Network Actually Represent? MNIST internals series — the practical companion to the ML notes (softmax & cross-entropy, backprop, training a 2-layer network). This time we stop deriving and start looking. Anyone who has trained a model eventually asks: The model is just a pile of numbers. In this lab’s case, exactly 7,860 of them (7,850 weights + 10 biases). What do those numbers actually mean? ...

Lab 1.2: Teach It 1s and 2s First, Then Introduce 3 — Watching Weights Reorganize MNIST internals series. In Lab 1.1 we established what a weight row is: a learned photograph of a class, contrast-carved against its rivals, painted by the update rule $W_c \leftarrow W_c - \eta\,\tfrac1N\sum (\hat p_c - y_c)\,x^\top$ . Now the experiment that motivated the series: start the model in an impoverished world — only 1s and 2s exist — then let it discover 3. This is continual learning in miniature, and it lets us watch, directly and quantitatively: ...