SIGN IN SIGN UP
huggingface / tokenizers UNCLAIMED

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

0 0 0 Rust

TAGS

20 tags
v0.20.2rc1

update python version install of setup python

v0.20.0rc1

Merge branch 'fix_release' of github.com:huggingface/tokenizers into branch_v0.20.0.rc1

v0.19.1rc0

"0.19.1-rc0" and Cargo.lock

v0.15.2

correct version for the release

v0.15.2rc1

add cargo lock `version = "0.15.2-rc1"`

v0.15.1.rc1

add cargo file for reproducibility

v0.15.0.rc1

add cargo file for reproducibility

v0.15.1rc0

pyo3: update to 0.20 (#1386) Co-authored-by: Mike Lui <mikelui@meta.com>

v0.15.1.rc0

pyo3: update to 0.20 (#1386) Co-authored-by: Mike Lui <mikelui@meta.com>

v0.14.1rc1

Preparing release candidate.