Computational Graphs, Part 3: A Single Neuron and Logistic Regression
Computational Graphs, Part 3: A Single Neuron and Logistic Regression The previous note showed how gradients add when one input feeds multiple operations. With that in place, we can now look at a real model: a single neuron. We will draw it as a graph, run the forward pass and backward pass by hand, and then connect it back to the logistic regression from the first note. 1. What you will learn How a single neuron is a small computational graph. The forward pass through a weighted sum and an activation function. The backward pass through the same graph. Why logistic regression is exactly a one-neuron network with sigmoid activation. How the cross-entropy loss fits into the graph as an extra node. Why the gradient formula from logistic regression matches the chain-rule result. 2. A single neuron A neuron with two inputs has three steps: ...