If I have a bziped tar file split over multiple dvd's what's the best way of extracting it without creating a lot of intermediate files? I don't want to first copy all the archive parts to an intermediate disk.

so far the best I can come up with is

terminal 1:

mkfifo temppipe1
mkfifo temppipe2
mkfifo temppipen
cat temppipe* | bunzip2 |tar -xvf -

terminal 2:

dd if=archive.tar.bz_part1 of=temppipe1
dd if=archive.tar.bz_partn of=temppipen

This works, allowing me to change the dvd after each dvd but strikes me as potentially creating a lot of named pipes, and potentially error prone. One typo in terminal 2 and tar is going to quit leaving no option but to start again. It's also essential to create enough names pipes at the beginning

Anyone got any better ideas?

  • You should look at the --multi-volume option - One of tar's specialties is splitting an archive across multiple devices sequentially. – Darth Android May 16 '13 at 18:45
  • Unfortunately, use of -M precludes the use of compression on all versions of tar (big down side), doesn't help at all extracting existing split archives, and has the an issue of portability. – camelccc May 16 '13 at 20:06

You can use a single pipe, and keep a writer open on it so it doesn't close until you are done.

terminal 1:

mkfifo temppipe
tar xvfj temppipe

terminal 2:

cat > temppipe

We won't have cat write anything to the pipe, but it's presence will keep the pipe open while we run the dd commands.

terminal 3:

dd if=archive.tar.bz_part1 of=temppipe # and so on

Then go back to terminal 2 and hit Control-D. cat will send the final end of file and tar will complete.

  • This looks very unix. I have a question though, I am now backing up things using BD 50G discs. I wonder if this will blow up the memory? – Carl Dong Nov 12 '15 at 2:20
  • No, memory usage will be constant. Only a limited amount of information can be sitting in the pipe waiting to be read. If dd tries to write data faster than tar can read it, it will block until tar catches up. – wingedsubmariner Nov 12 '15 at 20:43
  • Do you mean it is extracted on the fly as I pipe in the data? Also, I guess cat archive > temppipe also works, right? – Carl Dong Nov 12 '15 at 21:52
  • Yes, it will be extracted on the fly. And cat should work as well as dd. – wingedsubmariner Nov 13 '15 at 17:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.