Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I wanted to store a large video file on a FAT32 drive (~18 GB), but I discovered that this isn't simply possible due to limitations of the file system.

Is there a simple tool that exists for splitting the file into smaller parts that are storable, then reassembling them when I would like to retrieve the archived file?

Or is there a better way of storing large files on FAT32?

share|improve this question

migrated from stackoverflow.com Sep 22 '12 at 1:00

This question came from our site for professional and enthusiast programmers.

2  
Any archive utility (eg, 7zip) can do this, with or without compression. –  SLaks Sep 21 '12 at 3:05
    
Ah ok, good to know - I'll give it a shot. –  Alex Sep 21 '12 at 3:07
    
Ok, 7zip will do it - however it takes about 5-10 times longer than the script below. Anyway, I thought it might be helpful. –  Alex Sep 21 '12 at 3:14
    
Is this an external disk that needs to be mounted both on Windows and non-Windows systems? If not, why not just convert to a modern file system? Use whatever happens to be the default for the system that the drive is connected to. –  Isaac Rabinovitch Sep 22 '12 at 2:40
2  
> Ok, 7zip will do it - however it takes about 5-10 times longer than the script below. Even without compression? o.O –  Synetech Sep 22 '12 at 3:08

3 Answers 3

up vote 7 down vote accepted

Most file archivers, such as 7-Zip, WinZip, and WinRAR, allow you to split an archive across multiple files. If speed is important, you can try disabling the compression part of the program.

On GNU/Linux systems, you can use the split and cat programs from the coreutils package (e.g. split -b 4294967295 FOO /media/FATDISK/BAR to split FOO into BARaa, BARab, ... and cat /media/FATDISK/BAR?? > FOO to reassemble it). Mac OS X's command line split utility works in the same way.

share|improve this answer

I ended up writing a small Python 2 script to achieve this.


# Author: Alex Finkel 
# Email: alex@finkel.net

# This program splits a large binary file into smaller pieces, and can also 
# reassemble them into the original file.

# To split a file, it takes the name of the file, the name of an output 
# directory, and a number representing the number of desired pieces.

# To unsplit a file, it takes the name of a directory, and the name of an 
# output file.

from sys import exit, argv
from os import path, listdir

def split(file_to_split, output_directory, number_of_chunks):
    f = open(file_to_split, 'rb')
    assert path.isdir(output_directory)
    bytes_per_file = path.getsize(file_to_split)/int(number_of_chunks) + 1
    for i in range(1, int(number_of_chunks)+1):
        next_file = open(path.join(output_directory, str(i)), 'wb')
        next_file.write(f.read(bytes_per_file))
        next_file.close()
    f.close()

def unsplit(directory_name, output_name):
    assert path.isdir(directory_name)
    files = map(lambda x: str(x), sorted(map(lambda x: int(x), listdir(directory_name))))
    out = open(output_name, 'wb')
    for file in files:
        f = open(path.join(directory_name, file), 'rb')
        out.write(f.read())
        f.close()
    out.close()


if len(argv) == 4:
    split(argv[1], argv[2], argv[3])
elif len(argv) == 3:
    unsplit(argv[1], argv[2])
else:
    print "python split_large_file.py file_to_split output_directory number_of_chunks"
    print "python split_large_file.py directory name_of_output_file"
    exit()
share|improve this answer
    
-1 for reinventing the wheel. –  Mechanical snail Sep 22 '12 at 3:43
    
+1 for DIY. –  Tom Wijsman Sep 22 '12 at 3:50
    
+1 for sharing their work that might be useful to someone else in the future. Plus, if no-one ever re-invents wheels, one day no-one will know how to invent wheels. –  occulus Jan 26 at 12:18

Another option: use the split command from GNU Coreutils:

split --bytes=4G infile /media/FAT32drive/outprefix

to split the file into 4 GB chunks and save the chunks to the output drive.

The original file can be recovered by concatenating the chunks (with filenames sorted alphabetically).

For usage info, see the split manual.

Coreutils, including split, should be installed by default on Linux and Mac OS X. On Windows, it's available from GnuWin32, or from Cygwin.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.