Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

There's a host and a port(with TCP support) on which http service is running. Simple

wget host:port

doesn't return anything. Is it possible to get a list of all accessible files(html or txt) on that host(and the given port) which can be viewed?

Maybe there's some pattern like

wget host:port/*.[html|txt] or something

share|improve this question

migrated from Mar 18 '12 at 8:56

This question came from our site for information security professionals.

This command …

$ wget --spider DOWNLOAD-URL 

will check if DOWNLOAD-URL is available to download.

You can check this link for more options on wget: Ultimate wget guide. Also check the manual for wget you may more interesting options.

share|improve this answer
+1 for the link, but the question is about scanning the tree for the ALL public files which can be downloaded - the definite URIs are not known – Sergey Mar 17 '12 at 14:43
If there is no index, and you don't have list permissions on the host, it won't be possible. Otherwise, you want Section 10 (Mirror mode) in the linked guide. – EKW Mar 18 '12 at 9:36

Try using Google Search with the site: operator.

Example: I want to find out the URLs that are hosted on Solution: I go to Google Search and enter the search term Then, I use wget to begin spidering from all of the URLs found in the search results.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.