My goal is to download a single webpage to be fully functional offline in the same time it takes a browser to request and show the page. But that doesn’t seem to be happening with the command I am using.
The following command downloads a page and makes it fully functional offline, but it takes approximately 35 seconds where the hard refreshed browser requests and shows the page in about 5 seconds.
Can someone please help me understand why my
wget command is taking so much longer and how I can make it faster?
wget --page-requisites --span-hosts --convert-links --adjust-extension --execute robots=off --user-agent Mozilla --random-wait https://www.invisionapp.com/inside-design/essential-steps-designing-empathy/
More info and attempted solutions
- I removed
--random-waitbecause I thought it might be adding time for each file request, but this did nothing.
- I thought the https protocol might slow it down with extra calls back and forth for each file so I added
--no-check-certificate, but this did nothing.
- I read there could be an issue with IPv6 so I added
--inet4-only, but this did nothing.
- I read the DNS could slow things down so I added
--no-dns-cache, but this did nothing.
- I thought perhaps wget was downloading the assets sequentially one at a time so I tried to run multiple commands concurrently with between 3 and 16 threads/processes by removing
--no-clobberin the hopes that with multiple files would be downloaded at the same time and after all files were downloaded that I could run the command again removing
--convert-linksto make it fully functional offline, but this did nothing. I also thought that multiple threads would speed things up because it would remove the latency of the https checks by doing multiple at a time, but I didn't observe this.
- I read an article about running the command as root user in case there were any limits on a given user, but this did nothing.