Hello:
In my use-case I wish to calculate the number of tokens for a particular string of text without padding or truncation. However, I find that the tokenizer.json may by default have set to true for both. Is there a way to programmatically override it before encoding a text using this library?
Thanks