Large Language Models

Published

November 23, 2024

Modified

December 7, 2024

Inspiration

What is a Neural Network?

  1. Frank Rosenblatt’s Perceptron

  2. Deep Learning Networks

  • Input Layers
  • Output Layers
  • Hidden Layers
  • Activation
  1. Adaptation and Training
  • Backpropagation
  • Error Functions and Surfaces
  1. Working
  • “Repeated Weighted Averaging with Thresholding”
  • How does that end up “learning”? Is there an intuitive explanation?

Neural Nets in Code

Using torch.

References

  1. The Neural Network Zoo - The Asimov Institute. http://www.asimovinstitute.org/neural-network-zoo/

  2. It’s just a linear model: neural networks edition. https://lucy.shinyapps.io/neural-net-linear/

  3. Neural Network Playground. https://playground.tensorflow.org/

  4. Rohit Patel (20 Oct 2024). Understanding LLMs from Scratch Using Middle School Math: A self-contained, full explanation to inner workings of an LLM. https://towardsdatascience.com/understanding-llms-from-scratch-using-middle-school-math-e602d27ec876

  5. Machine Learning Tokyo: Interactive Tools for ML/DL, and Math. https://github.com/Machine-Learning-Tokyo/Interactive_Tool

  6. Anyone Can Learn AI Using This Blog. https://colab.research.google.com/drive/1g5fj7W6QMER4-03jtou7k1t7zMVE9TVt#scrollTo=V8Vq_6Q3zivl

  7. Neural Networks Visual with vcubingx

  1. Practical Deep Learning for Coders: An Online Free Course.https://course.fast.ai

Text Books

  1. Michael Nielsen. Neural Networks and Deep Learning, a free online book. http://neuralnetworksanddeeplearning.com/index.html

  2. Simone Scardapane. (2024) Alice’s Adventures in a differentiable Wonderland. https://www.sscardapane.it/alice-book/

Using R for DL

  1. David Selby (9 January 2018). Tea and Stats Blog. Building a neural network from scratch in R. https://selbydavid.com/2018/01/09/neural-network/

  2. torch for R: An open source machine learning framework based on PyTorch. https://torch.mlverse.org

  3. Akshaj Verma. (2020-07-24). Building A Neural Net from Scratch Using R - Part 1 and Part 2. https://rviews.rstudio.com/2020/07/20/shallow-neural-net-from-scratch-using-r-part-1/ and https://rviews.rstudio.com/2020/07/24/building-a-neural-net-from-scratch-using-r-part-2/

Maths

  1. Parr and Howard (2018). The Matrix Calculus You Need for Deep Learning.https://arxiv.org/abs/1802.01528
Back to top