SIGN IN SIGN UP
huggingface / tokenizers UNCLAIMED

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

0 0 182 Rust

Fixed vietnamese bug, now we have a thai bug.

N
Nicolas Patry committed
c0b9229833cfb3e67e2d42379f59228e24bcfe10
Parent: bcf35a2