Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a number of web sites I am archiving in order to retain many of the linked files there, specifically a number of PDFs.

I haven't had a problem using the Heritrix crawler to collect the sites. However I haven't found a good solution to extracting the files from these .warc files.

Does any one have experience with this, or have a preferred way to get these individual files out?

share|improve this question

2 Answers 2

I suggest to try warctools https://github.com/internetarchive/warctools it's python lib that is very easy to use.

share|improve this answer

I've used 7-Zip before to extract individual files or whole archives from Web Archive format files.

It's available from their site here.

share|improve this answer
    
Interesting. I'm on a Linux machine so I used the p7zip build. It doesn't seem to recognize the .warc as any sort of archive it can decompress (p7zip -d web-archive.warc). You were able to pull individual files out with 7-Zip though? –  walker Aug 9 '13 at 21:00
    
@walker I was indeed. Although the archive was not recognised, it did open with 7-Zip and the contents were displayed and were extracable. –  Martin Parkin Aug 9 '13 at 21:19
    
Hm. I've gotten onto a Windows machine and am using 7-Zip 9.20. I have three different .warc files but none are extractable by the program. Not sure what the problem is. –  walker Aug 22 '13 at 21:41

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.