Package profile
tokenizers
- Summary:
- Author: Nicolas Patry <patry.nicolas@protonmail.com>, Anthony Moi <anthony@huggingface.co>
- License: Apache Software License
- Homepage: https://github.com/huggingface/tokenizers
- Source: https://github.com/huggingface/tokenizers
- Number of releases: 103
- First release: 0.0.1 on 2019-11-01
- Latest release: 1.0.0rc2 on 2026-09-21
- Latest release size: 3.0 MB (wheel)
Dependencies
Tokenizers has 11 dependencies, 10 of which optional.View all 11 dependencies →Dependent packages
| Package | Optional | Group |
|---|---|---|
| ai2-olmo | false | |
| ai2-olmo-eval | false | |
| aleph-alpha-client | false | |
| chromadb | false | |
| cohere | false |