Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Is there a way to list all the downloadable files on a specific website (even at a specific level say)?

My typical issue is: An author puts a preprint toto.pdf on his website, then the paper gets published and the author removes (or redirects) the link to the file in his index.html, but keeps the file itself in the public folder.

If you are aware that the name was toto.pdf you still can download it, but what if you don't have the link (or if you have forgotten the name for instance)?

share|improve this question
add comment

3 Answers 3

up vote 1 down vote accepted

Google can help you,if you know the extension, in this case, pdf, use the query

filetype:pdf site:google.com

Replace google.com with the site in which you want to search for the pdf.

Example link

share|improve this answer
    
filetype:pdf site:google.com would provide more accurate results –  user1055604 Mar 18 '12 at 10:42
1  
@user1055604 done :) –  HackToHell Mar 18 '12 at 11:03
    
Thanks. I was more or less proceeding like this, but your way is more precise. But this is still not perfect : impossible to detect hidden .pdf files with this method. Moreover the google search surprisingly misses a lot of pdf files on some sites. –  Niels Mar 24 '12 at 8:58
add comment

You can try using Extreme Picture Finder to periodically download new PDFs (yes, it can download anything, not only pictures) automatically. You can even make this program work in pair with Windows Task Scheduler to automate the process completely. Here is the tutorial:

http://www.exisoftware.com/news/automatic-downloads-with-windows-task-scheduler.html

So, make it crawl the website once or twice a day with the option "Update" to get only new files and you won't miss anything.

share|improve this answer
    
thanks, but this seems windows-only ... –  Niels Apr 7 at 13:02
    
Yes, it's only for Windows. –  Egor Apr 10 at 9:58
add comment

It depends on how the HTTP server has been configured, which can be done (under most HTTPd's) on a per-directory level and/or globally.

If the server is configured to allow it, you can browse the contents of a given directory simply by entering it's URL into your address bar. The address of the directory is the same (usually, but let's leave URL rewriting aside) as the file's URL, sans the filename. For example, http://www.example.com/files/Form0123.pdf is located in the directory at http://www.example.com/files/. If the server is configured to do so it may present a default index page when a directory with no explicit index file is requested. The default index can, literally, be anything but is usually a listing of the directory's contents.

Browsing to a directory URL can yield any one of a number of results. First, you may get an error message, 404 or perhaps 403. In this case the server is configured to deny access to the directory default index if no index is specified for a directory. Other configurations will yield different results and the only way to know what will happen with a given URL is to try it and see.

share|improve this answer
    
thanks for the technical explanation. But it seems that browsing is not allowed by default, very often a .html file is loaded instead of the content of the directory. –  Niels Mar 24 '12 at 8:54
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.