Get all public files with wget

Question

There's a host and a port(with TCP support) on which http service is running. Simple

wget host:port

doesn't return anything. Is it possible to get a list of all accessible files(html or txt) on that host(and the given port) which can be viewed?

Maybe there's some pattern like

wget host:port/*.[html|txt] or something

Answer 1

This command …

$ wget --spider DOWNLOAD-URL 

will check if DOWNLOAD-URL is available to download.

You can check this link for more options on wget: Ultimate wget guide. Also check the manual for wget you may more interesting options.

Answer 2

Try using Google Search with the site: operator.

Example: I want to find out the URLs that are hosted on nlp.stanford.edu:8080. Solution: I go to Google Search and enter the search term site:nlp.stanford.edu:8080. Then, I use wget to begin spidering from all of the URLs found in the search results.