Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm looking for a solution between a proxy and wget, since neither fully addresses the issue I'm trying to solve.

I want to download files to a specific directory on my home machine (a Linux server - command-line only) from remote places (via smartphone, web browser, etc.). This sounds like a solution for wget, except many of the files I would download come from sources that create special download "sessions" for you via AJAX/Javascript (e.g. RapidShare.de, Hotfile.com).

I imagine I would have to use a proxy at some point in this process, but I don't want the download data to come to my remote device at all, since that would waste bandwidth. But, to use a proxy to access the sites and create the sessions would work for me.

So, solutions that don't work (out-of-the-box, at least):

  • Remote Desktop: Windows only
  • VNC: I only have command-line access; also, not so friendly with smartphones and the like
  • wget: download links generally only exist for a "session"; very difficult to emulate AJAX environment for sites
  • proxy: sends all data to remote client; mounting the remote directory would cause the data to take three hops (site to proxy, server/proxy to client, client to remote directory). I'm looking for a direct (site to remote directory) route for the download.

In my perfect world, I'm envisioning browsing to a site via a proxy, but then when the browser classifies something as a Download, it just downloads to the remote directory (or even opens a file Save dialog and lets me choose where to save it on the remote server). The solution doesn't have to have this level of simplicity, but hopefully this helps clarify the thrust of my question.

share|improve this question
add comment

3 Answers

up vote 1 down vote accepted

wget has options to save and load cookies. Sometimes depending on the site if you use wget to load the webpage that the link is on and tell it to save cookies, then make another request for the actual file and have it load the cookies it will work.

It's a two step process like this:

wget --cookies=on --keep-session-cookies --save-cookies=cookie.txt http://domain.com/page_with_link.html
wget --referer=http://domain.com/page_with_link.html --cookies=on --load-cookies=cookie.txt --keep-session-cookies --save-cookies=cookie.txt http://domain.com/some_file_to_download.zip
share|improve this answer
    
This could help. Most of the sites have an intricate Javascript procedures to generate the links, enter captchas, etc., so I would definitely have to make more wget calls, but it's a start. –  palswim May 18 '11 at 22:56
add comment

Navigate to the address and download the file with Links over a SSH connection. A GNU Screen could also be used to finish the download without having the SSH connection still open.

share|improve this answer
add comment

offhand the only software for downloading off one click hosters that you can run on a linux cli that i remember is ploughshare - its pretty much a dead simple, and handles loads of file hosts. Don't forget to run it in screen or tmux so you can detach the session rather than needing to keep the ssh session open

If you didn't mind a GUI i'd suggest looking at slimrat

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.