Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I've found myself having to use a Python script to access a web archive.

What I have is a 'megawarc' web archive file from http://archive.org/details/archiveteam-fanfiction-warc-11. I need to un-megawarc this, using the Python script found at https://github.com/alard/megawarc.

I'm trying to run the restore command, and I have the three files needed (FILE.warc.gz, FILE.tar, and FILE.json.gz) from the first link.

I have both Python 2.7 and 3.3 installed.

Update:

I've ran both this method:

python megawarc restore FILE

And this method:

Make sure you have the files megawarc and ordereddict.py in the same directory, with the files you want to convert. Rename the file megawarc to megawarc.py Open a Python console in this directory

Type the following code (line by line) :

import sys

sys.argv = ['megawarc','restore','FILE']

import megawarc

megawarc.main()using python 2.7, and this is what I get..

c:\Python27>python megawarc restore FILE

Traceback (most recent call last): File "megawarc", line 563, in main()

File "megawarc", line 552, in main mwr.process()

File "megawarc", line 460, in process self.process_entry(entry, tar_out)

File "megawarc", line 478, in process_entry entry["target"]["offset"], entry["target" ["size"])

File "megawarc", line 128, in copy_to_stream raise Exception("End of file: %d bytes expected, but %d bytes read." % (buf_size, l))

Exception: End of file: 4096 bytes expected, but 236 bytes read.

Is there something else I'm missing?

I have the following files all in C:\python27:

  • FILE.tar.megawarc.json.gz
  • FILE.tar.megawarc.tar
  • FILE.tar.megawarc.warc.gz
  • megawarc
  • ordereddict.py

Is this some type of corrupt file error? Some type of buffer error? Is there something I'm missing?

share|improve this question

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.