01Foundations
Neural Network
What a neural network actually is, explained through a lemonade stand. Understand weights, layers, and training without any biology or jargon.
Read →
21 articles on neural networks, training, evaluation, and modern AI architecture — written with analogies, code, and no unnecessary jargon.
What a neural network actually is, explained through a lemonade stand. Understand weights, layers, and training without any biology or jargon.
What a loss function measures and why it drives training. Covers MSE for regression and Cross-Entropy for classification with concrete examples.
How gradient descent finds the minimum of any function by stepping in the steepest downhill direction. The core algorithm behind all of machine learning.
What the learning rate controls and how to choose it. Too high and training explodes. Too low and it crawls. Learn the intuition with concrete numbers.
Why activation functions exist and what happens without them. ReLU, Sigmoid, Softmax, GELU — when to use each and why depth requires non-linearity.
The difference between an epoch, a batch, and an iteration — explained with a clear analogy. Understand exactly how data flows through training.
What overfitting and underfitting look like, how to diagnose them from the loss curve, and five specific fixes for each — with code.
Why you need three datasets, not one. The exact rules for splitting data correctly and why touching the test set early destroys your results.
What regularisation does and why you need it. L1, L2, and weight decay explained with the intuition of keeping weights honest and small.
How dropout prevents overfitting by randomly disabling neurons during training. The lottery ticket analogy that makes it click immediately.
What batch normalisation does, why it stabilises training, and the exact layer order to use it in. BatchNorm1d vs BatchNorm2d explained.
What precision, recall, and F1 score actually measure. When accuracy misleads you, what weighted F1 tells you, and how to read a classification report.
How words become vectors and why geometric distance captures meaning. Word2Vec intuition, pretrained embeddings, and why they matter for NLP.
What a token actually is and how tokenisers split text. Why token count matters for LLM cost and context windows, explained simply.
Why you never apply Softmax before CrossEntropyLoss in PyTorch. What Softmax outputs, how cross-entropy measures confidence, and the most common classification bug.
How attention lets every word look at every other word simultaneously. Query, Key, Value — the bank analogy that makes self-attention click.
The full transformer architecture from input to output — encoder, decoder, positional encoding, residual connections. How BERT, GPT, and Claude are built from the same blocks.
Why you don't train from scratch. How transfer learning reuses knowledge from massive pretraining and the four strategies from full fine-tuning to feature extraction.
How to adapt a pretrained model to your specific task. Learning rate schedules, freezing layers, LoRA, and the exact code to fine-tune a language model.
How RAG gives LLMs access to external knowledge without retraining. Vector stores, embedding similarity, and when to use RAG vs fine-tuning.
How RLHF teaches LLMs to follow instructions and be helpful. The reward model, PPO, and why this is what makes Claude behave the way it does.
More articles available on Substack.
View on Substack →