Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm downloading a large file that's not an archive, and I want to combine the tasks of downloading and decompressing.

How can I do them simultaneously?

share|improve this question

The question is tagged with curl but the answer only uses wget.

With curl it's a little easier than wget because it can request compression and decompress without piping (url truncated for clarity).

curl --compressed
share|improve this answer
Looks like it does the trick. Great answer @user23337 – Alec Sep 30 '13 at 15:50
up vote 2 down vote accepted


wget -O - -o /dev/null | bunzip2 > freebase-simple-topic-dump.tsv

Where bunzip2 is an unzipping command for your compression format of choice. It must support piped input. And the file must be a single compressed file, not an archive.

It uses wget to pipe the downloaded file to the unzipping application, outputting to the specified filename.

share|improve this answer
Technically, it doesn't do them simultaneously, it just performs the download via wget first and then pipes the results as a whole into bunzip. If you attempted to unzip a file that wasn't completely written you'd get an error indicating that the end of file was reached too soon. – MaQleod Oct 7 '12 at 4:20
@MaQleod Sorry, I'm pretty sure that's incorrect. Try running it without forwarding the output to a file; it starts printing straight away. – Alec Oct 7 '12 at 4:32
I agree with @Alec as it is also faster – Karussell Nov 3 '15 at 16:57

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.