How do you view a websites structure (similar to ftp) without getting the default index page? Basically, how do you browse a websites 'file and folder structure'?


Many websites prevent this and this kind of logic will not work on many sites. Websites developed using a MVC framework has a 'non traditional' way of storing the files.

For example, Ruby on Rails (a MVC framework) has a different folder for the different types of models in the site. You can use the routes.rb file to make it appear that files are located elsewhere in the directories, but it's masked with routing of their actual locations. (not extremely accurate, but gives a dumb-down version of what happens)

enter image description here

Update: To answer your question specifically,

If you're running apache on the website, you can edit your httpd.conf file and add

Options All Indexes FollowSymLinks MultiViews

However, if the website is not yours, then you will be limited as to what you can access. Even if you do manage to find a site structure, through http, would be unable to see any server side executed code.

  • 1
    If the website is not yours and you do not care about the server side executed stuff, you can check out webreaper.net which will basically go through and click on all of the links. Perhaps if you mentioned what you're trying to do with the site structure, we can point you in the proper direction.
    – kobaltz
    Oct 4 '11 at 15:36
  1. generate sitemap (lot of tools online)
  2. HTTP Graph plugin for Gephi
  3. Ms Visio (File menu, point to New, point to Software and Database, and then click Web Site Map.)
  4. Spidering (Wikto, etc..)
  5. Directory Traversal Attacks (OWASP, or the Open Web Application Security Project, developed a tool that is excellent for this purpose, named DirBuster.)
  6. URL Fuzzer - Discover hidden files and directories
  7. use Xenu link sleuth to map a website's links the way googlebot would
  8. sometimes site has a robots.txt or sitemap.xml file they can often give some information about directory structure but they likely won't list everything

It is not possible to get a listing of the file and folder structure of a website, unless the site owner specifically wants you to be able to see it (or has forgotten to prevent you from seeing it).

Websites are not like computers on your network they are remote machines and as such must have security in place. If they want you to see the structure you will see it, if they don't you won't its that simple.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy