9

I wanted to store a large video file on a FAT32 drive (~18 GB), but I discovered that this isn't simply possible due to limitations of the file system.

Is there a simple tool that exists for splitting the file into smaller parts that are storable, then reassembling them when I would like to retrieve the archived file?

Or is there a better way of storing large files on FAT32?

5
  • 2
    Any archive utility (eg, 7zip) can do this, with or without compression.
    – SLaks
    Sep 21, 2012 at 3:05
  • Ah ok, good to know - I'll give it a shot.
    – Alex
    Sep 21, 2012 at 3:07
  • Ok, 7zip will do it - however it takes about 5-10 times longer than the script below. Anyway, I thought it might be helpful.
    – Alex
    Sep 21, 2012 at 3:14
  • Is this an external disk that needs to be mounted both on Windows and non-Windows systems? If not, why not just convert to a modern file system? Use whatever happens to be the default for the system that the drive is connected to. Sep 22, 2012 at 2:40
  • 2
    > Ok, 7zip will do it - however it takes about 5-10 times longer than the script below. Even without compression? o.O
    – Synetech
    Sep 22, 2012 at 3:08

5 Answers 5

10

Most file archivers, such as 7-Zip, WinZip, and WinRAR, allow you to split an archive across multiple files. If speed is important, you can try disabling the compression part of the program.

On GNU/Linux systems, you can use the split and cat programs from the coreutils package (e.g. split -b 4294967295 FOO /media/FATDISK/BAR to split FOO into BARaa, BARab, ... and cat /media/FATDISK/BAR?? > FOO to reassemble it). Mac OS X's command line split utility works in the same way.

5

If you are looking for a quick solution to this, please see the other answers featuring 7zip or split. This is more of a fun solution.

I ended up writing a small Python 2 script to achieve this.


# Author: Alex Finkel 
# Email: [email protected]

# This program splits a large binary file into smaller pieces, and can also 
# reassemble them into the original file.

# To split a file, it takes the name of the file, the name of an output 
# directory, and a number representing the number of desired pieces.

# To unsplit a file, it takes the name of a directory, and the name of an 
# output file.

from sys import exit, argv
from os import path, listdir

def split(file_to_split, output_directory, number_of_chunks):
    f = open(file_to_split, 'rb')
    assert path.isdir(output_directory)
    bytes_per_file = path.getsize(file_to_split)/int(number_of_chunks) + 1
    for i in range(1, int(number_of_chunks)+1):
        next_file = open(path.join(output_directory, str(i)), 'wb')
        next_file.write(f.read(bytes_per_file))
        next_file.close()
    f.close()
                
def unsplit(directory_name, output_name):
    assert path.isdir(directory_name)
    files = map(lambda x: str(x), sorted(map(lambda x: int(x), listdir(directory_name))))
    out = open(output_name, 'wb')
    for file in files:
        f = open(path.join(directory_name, file), 'rb')
        out.write(f.read())
        f.close()
    out.close()
        
    
if len(argv) == 4:
    split(argv[1], argv[2], argv[3])
elif len(argv) == 3:
    unsplit(argv[1], argv[2])
else:
    print "python split_large_file.py file_to_split output_directory number_of_chunks"
    print "python split_large_file.py directory name_of_output_file"
    exit()
4
  • 3
    -1 for reinventing the wheel. Sep 22, 2012 at 3:43
  • +1 for DIY. Sep 22, 2012 at 3:50
  • 2
    +1 for sharing their work that might be useful to someone else in the future. Plus, if no-one ever re-invents wheels, one day no-one will know how to invent wheels.
    – occulus
    Jan 26, 2014 at 12:18
  • 1
    reinventing the wheel is good if one have time, but it's not the solution for a right-away result when you already have to tools available
    – phuclv
    Apr 25, 2017 at 15:18
4

Another option: use the split command from GNU Coreutils:

split --bytes=4G infile /media/FAT32drive/outprefix

to split the file into 4 GB chunks and save the chunks to the output drive.

The original file can be recovered by concatenating the chunks (with filenames sorted alphabetically).

For usage info, see the split manual.

Coreutils, including split, should be installed by default on Linux and Mac OS X. On Windows, it's available from GnuWin32, or from Cygwin.

2
  • great - worked with cygwin in windows - very straightforward - if you wanted to update with the recombine command that would be great, but I will look it up and suggest an edit if you don't!
    – tom
    Jan 9, 2015 at 4:13
  • commented too soon - for me I needed to put --bytes=3G else it failed as 4G hit the file size limit --- but with 3G it worked a treat (well)
    – tom
    Jan 9, 2015 at 4:29
3

I had exactly this problem and solved it using help from the answers already posted, but not one of them described the method exactly.

I used

split --bytes=3G infile /media/FAT32drive/outprefix

which save infile to multiple files named outprefixaa, outprefixab, outprefixac etc. with a maximum size of 3GB - when I tried to use 4GB on Fat 32 it failed.

to recombine the files I used

cat /media/FAT32drive/outprefix?? > infile

To give more specific details I did this in windows, but using a cygwin window. So the file path in my case for the external FAT32 harddisk was /cydrive/e/

0

For creating files with maximum allowed size on vfat (2³²-1 bytes), use the following command (in bash):

split --bytes=$((2**32-1)) infile /media/FAT32drive/outprefix

or, if you don't want to use bash's inline math:

split --bytes=4294967295 infile /media/FAT32drive/outprefix

The '--bytes=4G' fails, because 4G equals to 2³² bytes, which is exactly one byte more than the maximum file size on vFAT.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .