Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a number of web sites I am archiving in order to retain many of the linked files there, specifically a number of PDFs.

I haven't had a problem using the Heritrix crawler to collect the sites. However I haven't found a good solution to extracting the files from these .warc files.

Does any one have experience with this, or have a preferred way to get these individual files out?

share|improve this question

I suggest to try warctools https://github.com/internetarchive/warctools it's python lib that is very easy to use.

share|improve this answer

I've used 7-Zip before to extract individual files or whole archives from Web Archive format files.

It's available from their site here.

share|improve this answer
    
Interesting. I'm on a Linux machine so I used the p7zip build. It doesn't seem to recognize the .warc as any sort of archive it can decompress (p7zip -d web-archive.warc). You were able to pull individual files out with 7-Zip though? – wxs Aug 9 '13 at 21:00
    
@walker I was indeed. Although the archive was not recognised, it did open with 7-Zip and the contents were displayed and were extracable. – Martin Parkin Aug 9 '13 at 21:19
    
Hm. I've gotten onto a Windows machine and am using 7-Zip 9.20. I have three different .warc files but none are extractable by the program. Not sure what the problem is. – wxs Aug 22 '13 at 21:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .