Skip to content

serialization #2

@sugarme

Description

@sugarme

Serialization of the Tokenizer and all the parts (PreTokenizer, Normalizer, ...) using encoding/gob. It is now easy to save/load an entire tokenizer.

See: https://stackoverflow.com/questions/28020070/golang-serialize-and-deserialize-back

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions