this is part of a course in LLM and my college course on Soft computing. I am using Encode-Decoder Architeture. and transformer is built from scratch.
Check it out by running with a 64-bit version of Python (the one labeled x86-64)
Apt Versions to try:
Ubuntu os -> ubuntu mate 18 python -> 3.6.8 numpy -> 1.17.0
mac os -> 10.14.6 python -> 3.6.4 numpy -> 1.17.0
Reference:
- https://www.datacamp.com/tutorial/pytorch-tutorial-building-a-simple-neural-network-from-scratch
- "https://arxiv.org/abs/1706.03762 - "Attention Is All You Need"
- DeepLearning.AI LLM notes