Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a html file with several hundred links, a research database of mine, of a sort, collected over the years. What would be the easiest way to check which one of them are still alive ?

(importing in firefox is out of the question)

share|improve this question

1 Answer 1

up vote 4 down vote accepted

Use wget. Simple, scriptable, command-line, and available on your favorite platform, whether it's Unix-ish, Win*, Cygwin, etc (see Wikipedia for links to various versions). From the manpage:

--spider
When invoked with this option, Wget will behave as a Web spider, which means that it will not download the pages, just check that they are there. For example, you can use Wget to check your bookmarks:

wget --spider --force-html -i bookmarks.html

This feature needs much more work for Wget to get close to the functionality of real web spiders.

You might want the --no-verbose and/or --output-document=file options too.

share|improve this answer
    
probably doable with curl too. –  quack quixote May 10 '10 at 3:41
    
@quack why is the answer marked as CW ? –  Sathya May 10 '10 at 4:19
    
@sathya: why not? :) –  quack quixote May 10 '10 at 4:29
    
That will do, yes. Btw, I don't think this should be CW. –  ldigas May 10 '10 at 5:12
    
@idigas: honestly, it's CW because i believe this question's a duplicate. haven't managed to search up a predecessor yet tho. no biggie; if someone wants to add a curl example they can. –  quack quixote May 10 '10 at 5:46

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.