4

I have a html file with several hundred links, a research database of mine, of a sort, collected over the years. What would be the easiest way to check which one of them are still alive ?

(importing in firefox is out of the question)

1

1 Answer 1

5

Use wget. Simple, scriptable, command-line, and available on your favorite platform, whether it's Unix-ish, Win*, Cygwin, etc (see Wikipedia for links to various versions). From the manpage:

--spider
When invoked with this option, Wget will behave as a Web spider, which means that it will not download the pages, just check that they are there. For example, you can use Wget to check your bookmarks:

wget --spider --force-html -i bookmarks.html

This feature needs much more work for Wget to get close to the functionality of real web spiders.

You might want the --no-verbose and/or --output-document=file options too.

7
  • probably doable with curl too. May 10, 2010 at 3:41
  • @quack why is the answer marked as CW ?
    – Sathyajith Bhat
    May 10, 2010 at 4:19
  • @sathya: why not? :) May 10, 2010 at 4:29
  • That will do, yes. Btw, I don't think this should be CW.
    – Rook
    May 10, 2010 at 5:12
  • @idigas: honestly, it's CW because i believe this question's a duplicate. haven't managed to search up a predecessor yet tho. no biggie; if someone wants to add a curl example they can. May 10, 2010 at 5:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .