Package search
pyonmttok
- Summary: Fast and customizable text tokenization library with BPE and SentencePiece support
- Author: OpenNMT
- Homepage: https://opennmt.net
- Source: https://github.com/OpenNMT/Tokenizer/
- Repo profile
- Number of releases: 66
- First release: 1.3.0 on 2018-04-07T08:44:30
- Latest release: 1.37.1 on 2023-03-01T13:00:27
| Dependency | Optional |
|---|