Package profile
TokeNicer
- Summary: A (nicer) tokenizer you want to use for model `inference` and `training`: with all known peventable `gotchas` normalized or auto-fixed.
- Author: ModelCloud <qubitium@modelcloud.ai>
- License: Apache-2.0
- Homepage: https://github.com/ModelCloud/Tokenicer
- Source: https://github.com/ModelCloud/Tokenicer
- Number of releases: 15
- First release: 0.0.1.dev0 on 2025-02-08
- Latest release: 0.0.14 on 2026-07-22
- Latest release size: 13.9 KB (sdist)