Search results

cannot load Qwen3-235B-A22B tokenizer.json

Status: Open.

#75 In sugarme/tokenizer;

· yunqi123 opened on Dec 22, 2025

error: cannot find ' ' in the vocab.

Status: Open.

#73 In sugarme/tokenizer;

· 10196 opened on Nov 24, 2025

Error in Roberta post-processing when trimOffsets is false

Status: Open.

#66 In sugarme/tokenizer;

· season-studio opened on Sep 2, 2025

endOfWordSuffix does not take effect in BPE model

Status: Open.

#65 In sugarme/tokenizer;

· season-studio opened on Sep 2, 2025

Is it support Chinese?

Status: Open.

#53 In sugarme/tokenizer;

· wubin1989 opened on Mar 6, 2025

Support Embedding??

Status: Open.

#51 In sugarme/tokenizer;

· ErfanMomeniii opened on Jan 11, 2025

regexp dont support ?i

Status: Open.

#43 In sugarme/tokenizer;

· xuxiaoxia96 opened on Aug 6, 2024

Compile to wasm target

Status: Open.

#35 In sugarme/tokenizer;

· tiero opened on Oct 12, 2023

OpenAI CLIP tokenization support?

Status: Open.

#34 In sugarme/tokenizer;

· kristofmaar opened on Sep 12, 2023

int64

enhancementNew feature or requestNew feature or request

Status: Open.

#15 In sugarme/tokenizer;

· sugarme opened on Oct 22, 2020

serialization

enhancementNew feature or requestNew feature or request

Status: Open.

#2 In sugarme/tokenizer;

· sugarme opened on Sep 20, 2020