Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've found myself having to use a Python script to access a web archive.

What I have is a 'megawarc' web archive file from I need to un-megawarc this, using the Python script found at

I'm trying to run the restore command, and I have the three files needed (FILE.warc.gz, FILE.tar, and FILE.json.gz) from the first link.

I have both Python 2.7 and 3.3 installed.


I've ran both this method:

python megawarc restore FILE

And this method:

Make sure you have the files megawarc and in the same directory, with the files you want to convert. Rename the file megawarc to Open a Python console in this directory

Type the following code (line by line) :

import sys

sys.argv = ['megawarc','restore','FILE']

import megawarc

megawarc.main()using python 2.7, and this is what I get..

c:\Python27>python megawarc restore FILE

Traceback (most recent call last): File "megawarc", line 563, in main()

File "megawarc", line 552, in main mwr.process()

File "megawarc", line 460, in process self.process_entry(entry, tar_out)

File "megawarc", line 478, in process_entry entry["target"]["offset"], entry["target" ["size"])

File "megawarc", line 128, in copy_to_stream raise Exception("End of file: %d bytes expected, but %d bytes read." % (buf_size, l))

Exception: End of file: 4096 bytes expected, but 236 bytes read.

Is there something else I'm missing?

I have the following files all in C:\python27:

  • FILE.tar.megawarc.json.gz
  • FILE.tar.megawarc.tar
  • FILE.tar.megawarc.warc.gz
  • megawarc

Is this some type of corrupt file error? Some type of buffer error? Is there something I'm missing?

share|improve this question

You must log in to answer this question.

Browse other questions tagged .