Package profile
webdataset
- Summary: High performance storage and I/O for deep learning and data processing.
- Author: Thomas Breuel <tmbdev+removeme@gmail.com>
- License: BSD-3-Clause
- Homepage: http://github.com/webdataset/webdataset
- Number of releases: 79
- First release: 0.0.0 on 2020-02-20
- Latest release: 1.0.2 on 2025-06-19
- Latest release size: 73.2 KB (pure Python wheel)
Dependencies
Webdataset has 48 dependencies, 45 of which optional.Dependent packages
| Package | Optional | Group |
|---|---|---|
| megatron-energon | false | |
| salesforce-lavis | false | |
| aistore | true | pytorch |
| lhotse | true | webdataset |
| open-clip-torch | true | training |
Similar packages
- array-recordA file format that achieves a new frontier of IO efficiency
- pyiotoolsProvides several utilities for handling I/O
- iopathA library for providing I/O abstraction.
- tritonA language and compiler for custom Deep Learning operations
- onnxruntimeONNX Runtime is a runtime accelerator for Machine Learning models
- onnxruntime-gpuONNX Runtime is a runtime accelerator for Machine Learning models
- blosc2A fast & compressed ndarray library with a flexible compute engine.