Package profile
llmlingua
- Summary: To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
- Author: The LLMLingua team
- License: MIT License
- Homepage: https://github.com/microsoft/LLMLingua
- Source: https://github.com/microsoft/LLMLingua
- Number of releases: 14
- First release: 0.1.1 on 2023-10-08
- Latest release: 0.2.2 on 2024-04-09
- Latest release size: 29.8 KB (pure Python wheel)