Package search
smashed
- Summary: SMASHED is a toolkit designed to apply transformations to samples in datasets, such as fields extraction, tokenization, prompting, batching, and more. Supports datasets from Huggingface, torchdata iterables, or simple lists of dictionaries.
- Author:
- Homepage: https://github.com/allenai/smashed
- Source: https://github.com/allenai/smashed
- Repo profile
- Number of releases: 64
- First release: 0.1.0 on 2022-07-25T21:06:54
- Latest release: 0.21.5 on 2023-09-22T17:52:22
| Dependency | Optional |
|---|---|
| ftfy | false |
| glom | false |
| Jinja2 | false |
| necessary | false |
| numpy | false |
| platformdirs | false |
| trouting | false |
| autopep8 | true |
| black | true |
| blingfire | true |
| boto3 | true |
| datasets | true |
| dill | true |
| flake8 | true |
| flake8-pyi | true |
| Flake8-pyproject | true |
| ipdb | true |
| ipython | true |
| isort | true |
| moto | true |
| mypy | true |
| promptsource | true |
| pytest | true |
| smart-open | true |
| smashed | true |
| torch | true |
| torchdata | true |
| transformers | true |