PySpect

Home

lists

Frequently asked questions

© 2025 PySpect

Package profile

lm-eval

Summary: A framework for evaluating language models
Author: EleutherAI <contact@eleuther.ai>
Homepage: https://github.com/EleutherAI/lm-evaluation-harness
Source: https://github.com/EleutherAI/lm-evaluation-harness (Repo profile)
Number of releases: 12
First release: 0.0.1 on 2021-09-02
Latest release: 0.4.9 on 2025-06-19

Releases

PyPI Downloads

Dependencies

Lm-eval has 73 dependencies, 54 of which optional.

Dependencies of lm-eval (73).
Dependency	Optional
accelerate	false
datasets	false
dill	false
evaluate	false
jsonlines	false
more_itertools	false
numexpr	false
peft	false
pybind11	false
pytablewriter	false
rouge-score	false
sacrebleu	false
scikit-learn	false
sqlitedict	false
torch	false
tqdm-multiprocess	false
transformers	false
word2number	false
zstandard	false
aiohttp	true
antlr4-python3-runtime	true
auto-gptq	true
causal-conv1d	true
deepsparse-nightly	true
emoji	true
fugashi	true
fuzzywuzzy	true
gptqmodel	true
hf_transfer	true
ibm_watsonx_ai	true
immutabledict	true
jieba	true
kstar-planner	true
langdetect	true
lark	true
librosa	true
lm_eval	true
mamba_ssm	true
math_verify	true
mypy	true
nagisa	true
neologdn	true
nltk	true
numpy	true
optimum	true
pandas	true
pddl	true
pre-commit	true
promptsource	true
pycountry	true
pytest	true
pytest-cov	true
pytest-xdist	true
python-dotenv	true
requests	true
rouge	true
rouge_score	true
sae_lens	true
scipy	true
sentencepiece	true
soundfile	true
sparseml-nightly	true
sparsify	true
sympy	true
tarski	true
tenacity	true
tiktoken	true
tqdm	true
unitxt	true
vllm	true
wandb	true
wonderwords	true
zeno-client	true

Details