Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

This Bash script is fine to download just a few files, but I need to download over a hundred of them. VisualWGet is able to download a file of files, but is unable to rename them.

Does someone know of a Windows utility that would take a two-column text list where column 1 would contain the full URL to the file, and column 2 would contain the name to be used for renaming?

share|improve this question

4 Answers 4

up vote 0 down vote accepted

You can also use Download Accelerator Plus (DAP) to do both the downloading and renaming.

You just need to create a DAP list, import it into the app, select the folder you want the files saved in then let it do the work.

To create a DAP list, just open a text editor and enter the following for each file you wanna download:

<filelist_index><fileinfo>
    <filename>FILENAME.EXT</filename>
    <url>FULL_URL_TO_FILE</url>
</fileinfo></filelist_index>

Replace FILENAME.EXT with the filename you want (don't forget the file EXTENSION). Just use MS Excel to consolidate the code then copy-paste them to a text file and save it as a XXX.DAP file. Here's a sample spreadsheet: http://ge.tt/6sO1B2g

share|improve this answer
    
Thanks for the link. –  OverTheRainbow Apr 8 '11 at 17:43

Or, using windows powershell...

$folder = "D:\Output\"
$userAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:7.0.1) Gecko/20100101 Firefox/7.0.1"
$web = New-Object System.Net.WebClient
$web.Headers.Add("user-agent", $userAgent)
$input = "D:\todownload.txt"
$delimit = " "

Get-Content $input |
    Foreach-Object { 
        $line = $_.split($delimit);
        $destination_file = ([io.path]::getfilename($line[1]))
        "[" + $destination_file  + "]" + "Downloading " + $line[0]
        try {
            $target = join-path $folder $destination_file
            $web.DownloadFile($line[0], $target)
        } catch {
            $_.Exception.Message + ": " + $destination_file
            add-content (join-path $folder ("downloader" + ".log")) ("[" + $destination_file + "] " + $_.Exception.Message)
            add-content (join-path $folder ("downloader" + ".retry")) ($line -join " ")
        }
    }

The input file (todownload.txt) is in the format:

http://path.to.download.com/file.jpg 0001_NewFileName_file.jpg
share|improve this answer

You can give a shot to the Firefox extension DownThemAll!. It can download multiple links from a page, you can filter the download list and also apply a rename pattern.

share|improve this answer
    
Thanks but I need to turn eg. "419FD4E354AF7CA75D06" into "main.c", etc. so a simple rename pattern won't do. –  OverTheRainbow Apr 8 '11 at 12:53
    
I see. In that case you can (for example) write an extremely simple Perl script. There are two different Perl distribution on windows available. It would slurp the mapping file and then apply it on a directory. –  bandi Apr 8 '11 at 13:10
    
Looks like I'll have to write a web crawler. Thank you. –  OverTheRainbow Apr 8 '11 at 13:23

perl -MLWP::Simple -n -e '($u,$n)=split;getstore($u,$n)' filenames.txt

(Tested OK)

where filenames.txt contains something like

http://foo.com/a/b?parm=x   this.dat
http://bar.org//d/e/f.html  that.html

The same script in longer form

#!/usr/bin/perl
use strict;
use warnings;
use LWP::Simple;

open my $fh, '<', "filenames.txt" or die "can't read filenames.txt because $!\n";
while () {
  my ($url, $newname) = split;
  getstore($url, $newname);
}

(Untested)

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.