💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
TAGS
20 tags v0.20.2rc1
update python version install of setup python
v0.20.1rc1
update to ubuntu-22.04
v0.20.0rc1
Merge branch 'fix_release' of github.com:huggingface/tokenizers into branch_v0.20.0.rc1
v0.19.1rc0
"0.19.1-rc0" and Cargo.lock
v0.19.0rc0
Forcing Cargo.lock.
v0.16.0rc1
can't release py3.13
v0.15.2rc1
add cargo lock `version = "0.15.2-rc1"`
v0.15.1.rc1
add cargo file for reproducibility
v0.15.0.rc1
add cargo file for reproducibility
v0.15.1rc0
pyo3: update to 0.20 (#1386) Co-authored-by: Mike Lui <mikelui@meta.com>
v0.15.1.rc0
pyo3: update to 0.20 (#1386) Co-authored-by: Mike Lui <mikelui@meta.com>
v0.14.1rc1
Preparing release candidate.