Are there any tools in linux for splitting a file in place?

Question

I have a 500G file that I would like to split in linux, but I don't have another 500G free for running split(1).

Is there a tool or script that will split the file in-place into 1G chunks while using minimal extra space?

Answer 1

#!/bin/bash
# (c) whitequark 2010

set -e

if [ $# != 2 ]; then
  echo "Usage: $0 <filename> <part size>"
  echo "  This script will split file to multiple parts, starting from"
  echo "  the end, and truncating the original file in process."
  echo "  Part size is specified in bytes."
  echo "  Use at your own risk."
  exit 0
fi

filename=$1
partsize=$2

size=$(stat -c '%s' "${filename}")
parts=$(($size / $partsize))

do_split() {
  _part=$1
  _size=$2

  echo "Splitting part $_part"
  echo $(($partsize * ($_part - 1)))
  dd if="${filename}" of="${filename}.$(printf '%04d' $_part)" \
      count=1 bs=$partsize skip=$(($_part - 1))
  echo "Truncating source file"
  truncate "${filename}" --size="-$_size"
}

lastsize=$(($size % $partsize))
if [ $lastsize != 0 ]; then
  do_split $(($parts + 1)) $lastsize
fi

for i in $(seq $parts -1 1); do
  do_split $i $partsize
done

rm "${filename}"

gedit has successfully ran after disassembling and assembling it again.

Answer 2

Do you actually have the 500GB file yet? If you're generating a 500GB file by archiving a folder or disk, and then trying to split it, you can split it on-the-fly by piping the output of tar (or whatever you're using) into split:

sudo tar cvjsp /Volumes/BackupDisk/Backups.backupdb/ | \
     split -d -b 4480m - Backups.backupdb.tar.bz2.

This will make DVD-sized splits of an archive of my Time machine database. However, it does make them all at once, which means that it really doesn't do what you're looking for.

See my question here for more info. Whitequark's script could be useful over there with some slight modification! I'll have to try it.