SIGN IN SIGN UP
huggingface / tokenizers UNCLAIMED

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

0 0 0 Rust

COMMITS

/ bindings/python/pyproject.toml
python-313-release
March 12, 2024
November 10, 2023
November 6, 2023
M
Allow hf_hub 0.18 (#1383)
Mario Šaško committed
October 2, 2023
August 28, 2023
October 5, 2022
N
Updating python formatting. (#1079)
Nicolas Patry committed
May 24, 2021
November 27, 2020
P
Split Pre-Tokenizer (#542)
Patrick von Platen committed
September 23, 2020
November 27, 2019
A
Basic python setuptools
Anthony MOI committed