ayusrjn / llm-core-concepts Public

Notifications You must be signed in to change notification settings
Fork 0
Star 0

Implementation of mini BPE tokenizer, byte fallback, n-grams merges visualizer

0 stars 0 forks Branches Tags Activity

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 4 Commits
.ipynb_checkpoints		.ipynb_checkpoints
README.md		README.md
input.txt		input.txt
mini-BPE-tokenizer.py		mini-BPE-tokenizer.py
tiny_shakespear.pkl		tiny_shakespear.pkl
tokenization.md		tokenization.md
transformer_architecture.png		transformer_architecture.png
transformers.md		transformers.md

Repository files navigation

llm-core-concepts

Transformers Explained

About

Implementation of mini BPE tokenizer, byte fallback, n-grams merges visualizer

Report repository

Releases

No releases published

Packages

No packages published

Languages