18:T73e,

warc3-wet: Python3 library to work with WARC and WET files

Note: This is a fork of the original (now dead) warc repository.

WARC (Web ARChive) is a file format for storing web crawls.

This warc library makes it very easy to work with WARC files.::

import warc
with warc.open("test.warc") as f:
    for record in f:
        print(record['WARC-Target-URI'], record['Content-Length'])

And WET files.::

import warc
with warc.open("test.warc.wet") as f:
    for record in f:
        print(record['WARC-Target-URI'], record['Content-Length'])

Documentation

The documentation of the warc library is available at http://warc.readthedocs.org/.

Apart from the install from pip, which will not work for this warc3 version, the
interface as described there is unchanged.

This software is licensed under GPL v2. See LICENSE_ file for details.

Original Python2 Versions:

Python3 Port:

WET and seek support

0.2.4
Python 3.10 compatibility (thanks to @seanmacavaney)

0.2.3
Support seeking in WARC/WET

0.2.2
Allow WET parse

17:["$","div",null,{"className":"markdown-content max-w-2xl","dangerouslySetInnerHTML":{"__html":"$18"}}]