Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I am moving from an old ASP shopping cart site to a Drupal/Ubercart site. Part of this move is to ensure that old links will redirect to the new ones. To do that all I need is some way to get a list of all the links from the old site.

Preferably the results would have the page title and ideally I could give it some way to return other data from the page (ex. a CSS selector).

I would prefer if it were in OS X, but I can use Windows apps too.

I have tried Integrity, but it's output is nearly impossible to decipher, plus it doesn't seem to work well.

share|improve this question
R, can handle this. But I'm not sure how to do it for an entire website. Here's an example of parsing one page: stackoverflow.com/questions/3746256/… –  Brandon Bertelsen Aug 2 '12 at 6:44

1 Answer 1

If you don't mind writing Perl scripts ...

This module implements a configurable web traversal engine, for a robot or other web agent. Given an initial web page (URL), the Robot will get the contents of that page, and extract all links on the page, adding them to a list of URLs to visit.

share|improve this answer
I am horrible with Perl, and I cannot figure out how to install a module from CPAN =p –  Tyler Clendenin Aug 2 '12 at 15:38

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.