3

I googled for this but couldn't find an example of how to use eg. wget to download the directory structure of a website without downloading any of its files.

At this point, I just need to understand how a website is organized. I'll mirror the site later.

4
  • Do you mean the directory structure as in where the images, css, and js are stored?
    – Paul
    Apr 6, 2016 at 9:48
  • Yes, the directory structure, eg. /dir1, /dir2, etc. Apr 6, 2016 at 9:55
  • This isn't clear. Do you mean the directory structure of the assets that make up the site, such as where the images and css etc files reside, or do you mean the tree structure that describes how the pages link together (which is not related to directory structure). What for example would be in "dir1" in this case?
    – Paul
    Apr 6, 2016 at 13:28
  • This looks very relevant to web crawl the list of web pages on a particular web site. stackoverflow.com/questions/857653/…
    – simpleb
    Apr 6, 2016 at 14:32

1 Answer 1

3

At the command prompt type:

wget -r --spider www.your-website.com

Alternate command with an option to specify the maximum depth level depth.

wget -r --spider -l depth www.your-website.com

Recursive retrieval options:

-r  
--recursive
    Turn on recursive retrieving.    The default maximum depth is 5.

--spider
    Don't download anything

-l depth
--level=depth
    Specify recursion maximum depth level depth.
3
  • but this does download the files, too.
    – xtofl
    Oct 8, 2018 at 7:52
  • The --spider option of wget doesn't download anything except for creating the empty directory structure of the URL without downloading any files into the directories.
    – karel
    Oct 8, 2018 at 9:50
  • Somehow, I read over that argument. My bad.
    – xtofl
    Oct 9, 2018 at 6:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .