Search results

Short re-based script

Status: Open.

#34 In r-three/Tokenizers;

· gsaltintas opened on Jul 28, 2025

Add re pattern column to the dataset

Status: Open.

#33 In r-three/Tokenizers;

· gsaltintas opened on Jul 28, 2025

[Robustness] Create Farsi Tokenizer Robustness Dataset

data

Status: Open.

Task

#32 In r-three/Tokenizers;

· Malikeh97 opened on Jul 22, 2025

Estimate the frequency perturbations in the training datasets

Status: Open.

#31 In r-three/Tokenizers;

· gsaltintas opened on Jul 8, 2025

Implement Normalization for Different Tokenizers

ablation studies

Status: Open.

Task

#29 In r-three/Tokenizers;

· Malikeh97 opened on May 30, 2025

Setup Global MMLU Testing Pipeline

evaluation

Status: Open.

Task

#27 In r-three/Tokenizers;

· Malikeh97 opened on May 22, 2025

Setup the Pipeline for Perplexity Test

evaluation

Status: Open.

Task

#26 In r-three/Tokenizers;

· Malikeh97 opened on May 22, 2025

Create Figures for Downstream Performance vs Tokenizer Metrics

visualization

Status: Open.

#24 In r-three/Tokenizers;

· Malikeh97 opened on May 22, 2025

Create TinyBenchmarks for Too Large Benchmarks

evaluation

Status: Open.

#23 In r-three/Tokenizers;

· Malikeh97 opened on May 22, 2025

Implement the Tokenizer Metrics Script

evaluation

Status: Open.

#21 In r-three/Tokenizers;

· Malikeh97 opened on May 22, 2025

Final Proofreading

paper writeup

Status: Open.

#19 In r-three/Tokenizers;

· Malikeh97 opened on May 22, 2025

Conclusion

paper writeup

Status: Open.

#18 In r-three/Tokenizers;

· Malikeh97 opened on May 22, 2025