Repository info

tiktoken

• Summary: a summary of tiktoken
• Description: tiktoken is a fast BPE tokeniser for use with OpenAI's models.
• Stars: 15093
• Number of forks: 1121