💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
COMMITS
November 4, 2024
A
update python version install of setup python
Arthur Zucker committed
A
some windows cna't support 3.13 :(
Arthur Zucker committed
A
cargo lock
Arthur Zucker committed
A
oups should be a RC1 first
Arthur Zucker committed
A
v0.20.2 + cargo
Arthur Zucker committed
November 2, 2024
A
update python release
Arthur Zucker committed
November 1, 2024
T
Bump actions versions (#1669)
tinyboxvk committed
D
PyO3 0.22 (#1665)
Dimitris Iliopoulos committed
October 29, 2024
T
Bump actions/checkout to v4 (#1667)
tinyboxvk committed
M
Add safety comments (#1651)
Manish Goregaokar committed
October 25, 2024
S
Unsound call of `set_var` (#1664)
sftse committed
October 24, 2024
R
Arg name correction: auth_token -> token (#1621)
rravenel committed
October 14, 2024
R
Fix off-by-one error in tokenizer::normalizer::Range::len (#1638)
Ryan Landay committed
October 10, 2024
D
Bump cookie and express in /tokenizers/examples/unstable_wasm/www (#1648)
dependabot[bot] committed
A
push new dev version
Arthur Zucker committed
October 4, 2024
H
style: simplify string formatting for readability (#1632)
Hamir Mahal committed
October 1, 2024
A
Fix documentation build (#1642)
Arthur committed
D
Bump webpack in /tokenizers/examples/unstable_wasm/www (#1641)
dependabot[bot] committed
D
Bump send and express in /tokenizers/examples/unstable_wasm/www (#1631)
dependabot[bot] committed
D
Bump serve-static and express in /tokenizers/examples/unstable_wasm/www (#1630)
dependabot[bot] committed
D
Bump body-parser and express in /tokenizers/examples/unstable_wasm/www (#1629)
dependabot[bot] committed
A
[`ignore_merges`] Fix offsets (#1640)
Arthur committed
September 30, 2024
D
Bump actions/download-artifact from 3 to 4.1.7 in /.github/workflows (#1626)
dependabot[bot] committed
August 12, 2024
1
fix filelink (#1610)
152334H committed
August 9, 2024
A
Update README.md (#1608)
Arthur committed
August 8, 2024
A
update dev version 0.20.0
Arthur Zucker committed
N
Fix CI (#1607)
Nicolas Patry committed
N
Perf improvement 16% by removing offsets. (#1587)
Nicolas Patry committed
A
add deserialize for pre tokenizers (#1603)
Arthur committed