Andrej Karpathy Neural Networks: Zero to Hero (YouTube)

You should be comfortable with basic Python, but the series builds much of the neural-network machinery from small, readable components. Karpathy begins with micrograd, a compact automatic-differentiation engine, before moving toward language models.

Karpathy exposes scalar operations, gradients, backpropagation, and parameter updates so viewers can see what frameworks normally hide. Later material uses PyTorch, but only after deriving core mechanics in projects such as micrograd and makemore.

Makemore is a character-level language-modeling project that learns to generate new strings resembling a training list, commonly names. Karpathy uses it to progress from bigrams to multilayer perceptrons, recurrent networks, and transformer-style attention.

Yes, one lesson develops a small GPT-style language model from first principles, including causal self-attention and transformer blocks. It is educational code rather than a competitor to production systems such as ChatGPT, and it can be trained on a modest text dataset.