i have a web directory that has many folders and many sub folders containing files.
i need to download everything using wget or bash.
Super User is a question and answer site for computer enthusiasts and power users. It only takes a minute to sign up.Sign up to join this community
wget -r and see if that works.
The best way is:
wget -m <url>
Which is short for wget "mirror":
-m, --mirror shortcut for -N -r -l inf --no-remove-listing.
$ wget \ --recursive \ --no-clobber \ --page-requisites \ --html-extension \ --convert-links \ --restrict-file-names=windows \ --domains website.org \ --no-parent \ www.website.org/tutorials/html/
This command downloads the Web site www.website.org/tutorials/html/.
The options are:
Or try solution from ask Ubuntu.
wget --recursive (or whatever) didn't work for me (i'm on CentOS).
lftp did it:
lftp -c "open http://your.server/path/to/directory/; mirror"
wget -r -l 5 -O whatever http://example.com/
You have a web directory? Is it situated on a remote machine and you can only access it through HTTP, or do you have shell access? Your mention of bash implies shell access, unless you mean using wget from the bash prompt.
Wget is not always very efficient so if you have shell access to the machine where the web directory is located and you want to download it, you could do this
$ tar cjf webdir.tar.bz2 webdir
and then transfer the archive with ftp or scp.
You could also try the following if you have an FTP account:
lftp USER:PASSWORD@FTPSERVER -e "mirror&&exit"