This is a personal notebook to learn and experiment theories in the deep-learning space from math to code. Everything is designed to be simple enough to be easily understandable with basic knowledge in linear algebra and Python. Also all the code is written in Python using tinygrad.
- Good understanding of linear algebra (matrices, derivative, vectors)
- Understand basics of Pythonand programming
I recommend you to clone the repository and open the vault in Obsidian. You can still read it all through the GitHub Web frontend but it's a far worse experience.
Then you should install the packages by running uv
I've also written a single line bash script to launch the virtual environment easily.
You can check in the md/ directory to find the documents linked with some scripts.
python3 -m scripts.attention
python3 -m scripts.experimentation.attention