Project 1 for Subword Modeling

All of the work is reproducible in tokenization.ipynb

Name		Name	Last commit message	Last commit date
Latest commit History 13 Commits
Baselines		Baselines
__pycache__		__pycache__
autoencoder_predictions		autoencoder_predictions
conditional_entropy_predictions		conditional_entropy_predictions
dataset		dataset
segmenter_predictions		segmenter_predictions
transformer_predictions		transformer_predictions
.gitignore		.gitignore
README.md		README.md
eval.py		eval.py
preprocess.py		preprocess.py
tokenization.ipynb		tokenization.ipynb
tokenization2.ipynb		tokenization2.ipynb

Provide feedback