Numpy

Lab 1.3: From Templates to Features — What One Hidden Layer Does to the Weights MNIST internals series. Lab 1.1 gave us the clean answer: in a linear softmax model, a weight row is a matched-filter photograph of its class. Lab 1.2 showed those photographs being painted, re-carved, and out-bid. Now the question this series was really building toward: add one hidden layer. What happens to the weight picture? If you’ve watched 3Blue1Brown’s neural nets series, you might expect: “layers learn edges, then curves, then parts; visualizing raw weights won’t tell you much here.” We can now test that ourselves instead of taking it on faith, and the truth lands in a more interesting place: ...

Lab 2.1: How Word Vectors Learn to Mean — Embeddings From Zero, and What They Secretly Compute Language-model internals series. In the MNIST series we took apart what weights represent for vision. New series, same ritual — build it from scratch, check gradients numerically, claim nothing without a baseline or a measured number. The question for this lab, in the plainest possible form: A word becomes a row of numbers. How can those numbers possibly mean anything? ...

Lab 2.2: A Language Model Without Attention — char-level RNN, backprop through time, and the measured death of gradient signal Language-model internals series. Lab 2.1 gave words geometry without any sequence at all — windows don’t know order. This lab puts order back, the pre-2017 way: a recurrent network that reads one character at a time and carries everything it remembers in one vector. The organizing question of this series, sharpened: ...

Lab 1.1: What Do the Weights of a Neural Network Actually Represent? MNIST internals series — the practical companion to the ML notes (softmax & cross-entropy, backprop, training a 2-layer network). This time we stop deriving and start looking. Anyone who has trained a model eventually asks: The model is just a pile of numbers. In this lab’s case, exactly 7,860 of them (7,850 weights + 10 biases). What do those numbers actually mean? ...

Training a 2-Layer Network in NumPy: From Scalar to Vectorized Backprop The previous note did the backward pass for a small 2-layer network by hand, one scalar at a time. That is the best way to understand what backprop actually does. This note takes the next step: turn that scalar walk into compact, vectorized NumPy code. The math is unchanged; the only thing that changes is notation. I will introduce every matrix slowly — what its rows and columns mean, where the division by the batch size comes from, and why it is exactly the same algorithm you already did by hand. ...

ML Refresher: Linear and Logistic Regression This is the first note in the ML → Deep Learning → Transformers → LLMs series. The goal is to rebuild working memory of the basics before we get to neural networks: what a model is, how a loss function measures error, and how gradient descent tunes parameters. We will implement linear regression and logistic regression from scratch in NumPy, then compare with scikit-learn. If the code and gradients feel obvious, you are ready for the next note (computational graphs and backprop). If not, this is exactly the foundation to lock down first. ...