I'm looking for a way to compress a large file (~10GB) into several files that wont exceed 150MB each.
Super User is a question and answer site for computer enthusiasts and power users. It only takes a minute to sign up.Sign up to join this community
This will compress file /path/to/your/large/file and creates many files with the prefix compressed.gz in the current directory, each file with a maximum size of 150000000 bytes:
gzip -c /path/to/your/large/file | split -b 150000000 - compressed.gz
To uncompress the file resulting in the uncompressed file "/path/to/decrompressed/file" compressed using the command above use:
cat compressed.gz* | zcat > /path/to/decrompressed/file
split [OPTION] [INPUT [PREFIX] - split a file into pieces
Output fixed-size pieces of INPUT to PREFIXaa, PREFIXab, ...; default size is 1000 lines, and default PREFIX is 'x'. With no INPUT, or when INPUT is -, read standard input.
SIZE may have a multiplier suffix: b for 512, k for 1K, m for 1 Meg.