Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a tar file that I want to split into multiple smaller tar files. This would be easy with split, but I want the splitted files to be fully usable tar files themselves, which split can't do as it will split at arbitrary points, not at file boundaries.

So how to split a tar file into smaller parts at file boundaries, so that no file ends up being half in one tar and half in the other tar?

Solutions that don't use tar and accomplish the task by other means would be welcome as well.

PS: Yes, there will be cases where this isn't possible (tar with files larger then the split size).

share|improve this question
    
[ star ](cdrecord.berlios.de/private/man/star/star.1.html) has a promising option in tsize=, but I did not see anything like [ bsdtar ](code.google.com/p/libarchive/wiki/ManPageBsdtar1)’s @archive that might complete the task. –  Chris Johnsen Sep 17 '10 at 11:58

2 Answers 2

up vote 3 down vote accepted

If recreating the archive is an option this Bash script should do the trick (it's just a possible manner):

#!/bin/bash

if [ $# != 3 ] ; then
    echo -e "$0 in out max\n"
    echo -e "\tin:  input directory"
    echo -e "\tout: output directory"
    echo -e "\tmax: split size threshold in bytes"
    exit
fi

IN=$1 OUT=$2 MAX=$3 SEQ=0 TOT=0
find $IN -type f |
while read i ; do du -bs "$i" ; done |
sort -n |
while read SIZE NAME ; do
    if [ $TOT != 0 ] && [ $((TOT+SIZE)) -gt $MAX ] ; then
        SEQ=$((SEQ+1)) TOT=0
    fi
    TOT=$((TOT+SIZE))
    TAR=$OUT/$(printf '%08d' $SEQ).tar
    tar rf $TAR "$NAME"
done

It sorts (ascending order) all the files by size and starts creating the archives; it switches to another when the size exceeds the threshold.

NOTE: Make you sure that the output directory is empty.

USE AT YOUR OWN RISK

share|improve this answer

I don't believe there are any existing tools to do this, but it would be reasonably easy to implement yourself. The tar format is pretty simple, so you'd just have to have a split that took it into consideration. The basic theory is to read a header, look at the stated length of the incoming file, and determine whether to split now or write out the current file. Read the next header, and repeat.

share|improve this answer
    
libarchive might be handy for something like this. –  Chris Johnsen Sep 17 '10 at 11:36

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.