-2

Is there a way to retrieve the source code of every webpage in a website and save them to a single text file? With the appropriate CRFL characters just as if the pages were retrieved individually under Internet Explorer? I've tried Wget, but can't seem to find the right combination of options.

2
  • Not generally, no. The source code may not be available for download, depending on the specific language and options used on the website. You can't get the source code to amazon, for example. Aug 4, 2015 at 19:05
  • Even with wget, you can only download where you have hyperlinks to follow.
    – barlop
    Aug 4, 2015 at 19:11

1 Answer 1

0

If the webserver has been configured correctly, no, this is not possible. This is because if you were able to download the source code of websites, you would be able to retrieve valuable data such as the website's database authentication details.

You may however download the rendered HTML of the webpage, as it appears in your Internet browser, with wget:

wget -E -H -k -K -p URLHERE

However some webservers may treat wget as a robot (a program that scrapes and saves data from web-pages), in this event you may use the -e robots=off switches in your wget command.

0

Not the answer you're looking for? Browse other questions tagged .