Package profile
pytokens
- Summary: A Fast, spec compliant Python 3.14+ tokenizer that runs on older Pythons.
- Author: Tushar Sadhwani <tushar.sadhwani000@gmail.com>
- License: MIT License
- Homepage: https://github.com/tusharsadhwani/pytokens
- Source: https://github.com/tusharsadhwani/pytokens
- Number of releases: 16
- First release: 0.0.0 on 2024-12-13
- Latest release: 0.4.1 on 2026-01-30
- Latest release size: 13.4 KB (pure Python wheel)
Dependencies
Pytokens has 9 dependencies, 9 of which optional.View all 9 dependencies →Dependent packages
Similar packages
- json-stream-rs-tokenizerA faster tokenizer for the json-stream Python library
- jwtJSON Web Token library for Python 3.
- toonsA high-performance TOON (Token Oriented Object Notation) parser and serializer for Python, implemented in Rust.
- mistletoeA fast, extensible Markdown parser in pure Python.
- libcstA concrete syntax tree with AST-like properties for Python 3.0 through 3.14 programs.
- srslyModern high-performance serialization utilities for Python
- flpcA Lightning Fast ⚡ Rust-based regex crate wrapper for Python3 to get faster performance. 👾