up vote 2 down vote favorite
1
share [g+] share [fb]

I have a html file with several hundred links, a research database of mine, of a sort, collected over the years. What would be the easiest way to check which one of them are still alive ?

(importing in firefox is out of the question)

link|improve this question
63% accept rate
feedback

1 Answer

up vote 4 down vote accepted

Use wget. Simple, scriptable, command-line, and available on your favorite platform, whether it's Unix-ish, Win*, Cygwin, etc (see Wikipedia for links to various versions). From the manpage:

--spider
When invoked with this option, Wget will behave as a Web spider, which means that it will not download the pages, just check that they are there. For example, you can use Wget to check your bookmarks:

wget --spider --force-html -i bookmarks.html

This feature needs much more work for Wget to get close to the functionality of real web spiders.

You might want the --no-verbose and/or --output-document=file options too.

link|improve this answer
probably doable with curl too. – quack quixote May 10 '10 at 3:41
@quack why is the answer marked as CW ? – Sathya♦ May 10 '10 at 4:19
@sathya: why not? :) – quack quixote May 10 '10 at 4:29
That will do, yes. Btw, I don't think this should be CW. – ldigas May 10 '10 at 5:12
@idigas: honestly, it's CW because i believe this question's a duplicate. haven't managed to search up a predecessor yet tho. no biggie; if someone wants to add a curl example they can. – quack quixote May 10 '10 at 5:46
show 2 more comments
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.