bindings/python - huggingface/tokenizers - Morph

SIGN IN SIGN UP

huggingface / tokenizers UNCLAIMED

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

0 0 0 Rust

	.cargo
	benches
	examples
	py_src
	scripts
	src
	tests
	.gitignore	5 B
	Cargo.lock	41.5 KB
	Cargo.toml	876 B
	CHANGELOG.md	22.4 KB
	conftest.py	576 B
	Makefile	908 B
	MANIFEST.in	188 B
	pyproject.toml	1.9 KB
	README.md	4.9 KB
	rust-toolchain	7 B
	setup.cfg	875 B
	stub.py	5.3 KB
	test.txt	1.6 KB