This project is a work in progress.
This list is a non-exhaustive list of all the things that needs to be done before this project is relatively feature complete.
- Implement embedding / deembedding compilation
- Separate tokenizer from embedding
- Export to ONNX
Recently complete:
- Implement attn compilation in attention sublayer
- Refactor compilation scoring
- Include compilation statistics (number of parameters, efficiency, etc)
- Better pass-through support for attention layers; We shouldn't require one head per zero.