Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I wrote a script, so I can automatically download from a hotfile using LWP::UserAgent. I've managed to get the file to download properly. How do I get the file size of the file before downloading?

I need this to display hot much of the file has been downloaded.

share|improve this question
add comment

3 Answers 3

up vote 3 down vote accepted

Here is a short snippet that does this.

use LWP::UserAgent;

sub GetFileSize{
    my $url=shift;
    $ua = new LWP::UserAgent;
    $ua->agent("Mozilla/5.0");
    my $req = new HTTP::Request 'HEAD' => $url;
    $req->header('Accept' => 'text/html');
    $res = $ua->request($req);
    if ($res->is_success) {
        my $headers = $res->headers;
        return $headers;
    }
    return 0;
}

$link = 'http://www.domain.com/anyfile.zip';
$header = GetFileSize($link);

print "File size: " . $header->content_length . " bytes\n";
print "Last moified: " . localtime($header->last_modified) . "\n";
exit;

Source

share|improve this answer
add comment

The above answer by Nifle helped me a lot. I just like to add that for an authenticated HTTP site, you need to add just one line, $req->authorization_basic( "$name", "$pwd" ); And it works fine.

sub GetFileSize{
    my $url=shift;
    my $name = shift;
    my $pwd = shift;

    my $browser = LWP::UserAgent->new;
    $browser->agent("Mozilla/5.0");
    my $req =  HTTP::Request->new( HEAD => "$url");
    $req->authorization_basic( "$name", "$pwd" );

    $req->header('Accept' => 'text/html');
    my $res = $browser->request($req);
    if ($res->is_success) {
        my $headers = $res->headers;
        return $headers->content_length;
    }
    return 0;
}
share|improve this answer
add comment

Just getting the size is really simple:

use LWP::Simple;

my $url='http://www.superuser.com/favicon.ico';
my ($type, $size) = head($url) or die "ERROR $url: $!";
print "$url: type: $type, size: $size$/";

.

However, for a real progress indicator you have to register a callback, like so:

use LWP::UserAgent;

my $url='http://superuser.com/questions/200468/how-to-get-file-size-before-downloading-the-file-lwpuseragent?rq=1';#'http://www.superuser.com/favicon.ico';
my $ttlDown = 0;
my $resp = LWP::UserAgent->new()->get($url, ':content_cb' => sub {
      my ($data, $response) = @_;
      my $size = $response->content_length;
      $ttlDown += length $data;
      printf("%7.1f KB of %7.1f (%5.1f%%)$/", 
        $ttlDown / 1024.0, $size / 1024.0, $ttlDown * 100.0 / $size
      );
      ##### TODO: append $data to file #####
});
print "$/-----$/".$resp->as_string();

Note the ##### TODO line: you might want to write the received bytes to disk there.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.