Utility to download and rename a bunch of files?

Question

This Bash script is fine to download just a few files, but I need to download over a hundred of them. VisualWGet is able to download a file of files, but is unable to rename them.

Does someone know of a Windows utility that would take a two-column text list where column 1 would contain the full URL to the file, and column 2 would contain the name to be used for renaming?

Answer 1

You can also use Download Accelerator Plus (DAP) to do both the downloading and renaming.

You just need to create a DAP list, import it into the app, select the folder you want the files saved in then let it do the work.

To create a DAP list, just open a text editor and enter the following for each file you wanna download:

<filelist_index><fileinfo>
    <filename>FILENAME.EXT</filename>
    <url>FULL_URL_TO_FILE</url>
</fileinfo></filelist_index>

Replace FILENAME.EXT with the filename you want (don't forget the file EXTENSION). Just use MS Excel to consolidate the code then copy-paste them to a text file and save it as a XXX.DAP file. Here's a sample spreadsheet: http://ge.tt/6sO1B2g

Answer 2

Or, using windows powershell...

$folder = "D:\Output\"
$userAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:7.0.1) Gecko/20100101 Firefox/7.0.1"
$web = New-Object System.Net.WebClient
$web.Headers.Add("user-agent", $userAgent)
$input = "D:\todownload.txt"
$delimit = " "

Get-Content $input |
    Foreach-Object { 
        $line = $_.split($delimit);
        $destination_file = ([io.path]::getfilename($line[1]))
        "[" + $destination_file  + "]" + "Downloading " + $line[0]
        try {
            $target = join-path $folder $destination_file
            $web.DownloadFile($line[0], $target)
        } catch {
            $_.Exception.Message + ": " + $destination_file
            add-content (join-path $folder ("downloader" + ".log")) ("[" + $destination_file + "] " + $_.Exception.Message)
            add-content (join-path $folder ("downloader" + ".retry")) ($line -join " ")
        }
    }

The input file (todownload.txt) is in the format:

http://path.to.download.com/file.jpg 0001_NewFileName_file.jpg

Answer 3

You can give a shot to the Firefox extension DownThemAll!. It can download multiple links from a page, you can filter the download list and also apply a rename pattern.

Answer 4

perl -MLWP::Simple -n -e '($u,$n)=split;getstore($u,$n)' filenames.txt

(Tested OK)

where filenames.txt contains something like

http://foo.com/a/b?parm=x   this.dat
http://bar.org//d/e/f.html  that.html

The same script in longer form

#!/usr/bin/perl
use strict;
use warnings;
use LWP::Simple;

open my $fh, '<', "filenames.txt" or die "can't read filenames.txt because $!\n";
while () {
  my ($url, $newname) = split;
  getstore($url, $newname);
}

(Untested)