Package profile
llmlingua
- Summary: To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
- Author: The LLMLingua team
- Homepage: https://github.com/microsoft/LLMLingua
- Source: https://github.com/microsoft/LLMLingua (Repo profile)
- Number of releases: 14
- First release: 0.1.1 on 2023-10-08
- Latest release: 0.2.2 on 2024-04-09
Releases
PyPI Downloads
Dependencies
Llmlingua has 12 dependencies, 6 of which optional.| Dependency | Optional |
|---|---|
| accelerate | false |
| nltk | false |
| numpy | false |
| tiktoken | false |
| torch | false |
| transformers | false |
| black | true |
| flake8 | true |
| isort | true |
| pre-commit | true |
| pytest | true |
| pytest-xdist | true |