https://sankethbk.github.io/blog/tags/features/Recent content in Features on Sanketh's BlogHugo -- 0.169.0en-usTue, 27 Oct 2026 00:00:00 +0530https://sankethbk.github.io/blog/posts/ml/labs/2026-10-27-lab-3-from-templates-to-features/Tue, 27 Oct 2026 00:00:00 +0530https://sankethbk.github.io/blog/posts/ml/labs/2026-10-27-lab-3-from-templates-to-features/<h1 id="lab-13-from-templates-to-features--what-one-hidden-layer-does-to-the-weights">Lab 1.3: From Templates to Features — What One Hidden Layer Does to the Weights</h1> <p><strong>MNIST internals series.</strong> <a href="https://sankethbk.github.io/blog/posts/ml/labs/2026-10-19-lab-1-mnist-from-pixels-to-neural-network/">Lab 1.1</a> gave us the clean answer: in a linear softmax model, a weight row is a matched-filter photograph of its class. <a href="https://sankethbk.github.io/blog/posts/ml/labs/2026-10-19-lab-2-continual-learning-1-2-then-3/">Lab 1.2</a> showed those photographs being painted, re-carved, and out-bid.</p> <p>Now the question this series was really building toward: <strong>add one hidden layer. What happens to the weight picture?</strong></p> <p>If you&rsquo;ve watched 3Blue1Brown&rsquo;s neural nets series, you might expect: &ldquo;layers learn edges, then curves, then parts; visualizing raw weights won&rsquo;t tell you much here.&rdquo; We can now test that ourselves instead of taking it on faith, and the truth lands in a more interesting place:</p>