Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a 500G file that I would like to split in linux, but I don't have another 500G free for running split(1).

Is there a tool or script that will split the file in-place into 1G chunks while using minimal extra space?

share|improve this question

2 Answers 2

up vote 9 down vote accepted
#!/bin/bash
# (c) whitequark 2010

set -e

if [ $# != 2 ]; then
  echo "Usage: $0 <filename> <part size>"
  echo "  This script will split file to multiple parts, starting from"
  echo "  the end, and truncating the original file in process."
  echo "  Part size is specified in bytes."
  echo "  Use at your own risk."
  exit 0
fi

filename=$1
partsize=$2

size=$(stat -c '%s' "${filename}")
parts=$(($size / $partsize))

do_split() {
  _part=$1
  _size=$2

  echo "Splitting part $_part"
  echo $(($partsize * ($_part - 1)))
  dd if="${filename}" of="${filename}.$(printf '%04d' $_part)" \
      count=1 bs=$partsize skip=$(($_part - 1))
  echo "Truncating source file"
  truncate "${filename}" --size="-$_size"
}

lastsize=$(($size % $partsize))
if [ $lastsize != 0 ]; then
  do_split $(($parts + 1)) $lastsize
fi

for i in $(seq $parts -1 1); do
  do_split $i $partsize
done

rm "${filename}"

gedit has successfully ran after disassembling and assembling it again.

share|improve this answer
2  
note that you can use 'dd if=/dev/null of="${filename}" seek=1 bs=$(($size - $_size))' if for some reason you don't have truncate –  ngoozeff Aug 19 '10 at 12:09
1  
+1 for the nice hack –  Aaron Digulla Aug 19 '10 at 14:09

Do you actually have the 500GB file yet? If you're generating a 500GB file by archiving a folder or disk, and then trying to split it, you can split it on-the-fly by piping the output of tar (or whatever you're using) into split:

sudo tar cvjsp /Volumes/BackupDisk/Backups.backupdb/ | \
     split -d -b 4480m - Backups.backupdb.tar.bz2.

This will make DVD-sized splits of an archive of my Time machine database. However, it does make them all at once, which means that it really doesn't do what you're looking for.

See my question here for more info. Whitequark's script could be useful over there with some slight modification! I'll have to try it.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.