Building a Large Language Model from Scratch

This project implements a basic transformer-based language model from scratch.

Components

1. Tokenization
2. Embedding Layer
3. Positional Encoding
4. Self-Attention
5. Transformer Block
6. Full Language Model

Usage

1. Install requirements: pip install -r requirements.txt
2. Train model: python train.py
3. Run predictions: python predict.py