I have some huge files on my mac, and the disk is almost filled. I would like to transfer the data using a usb flash drive to another computer that has terabytes of free space. How do I do that using a usb flash drive, which is too small to contain the whole file?
This will not be a step by step instruction. I don't know, if dd needs to be installed on a mac, on linux it's mostly available ootb, as it's very powerful.
You have to insert your thumbdrive, let's assume, it's mounted on /media/thumbdrive, and your big file is in /home/user/bigfile.img
Create file number one:
dd if=/home/user/bigfile.img of=/media/thumbdrive/bigfile.img.001 bs=1024 count=30G
dd if=/home/user/bigfile.img of=/media/thumbdrive/bigfile.img.001 bs=1024 count=30G skip=30G dd if=/home/user/bigfile.img of=/media/thumbdrive/bigfile.img.002 bs=1024 count=30G skip=60G
Put it all together into one folder on the target machine, and run
cat bigfile.img.* > bigfile.img
Might need some minor changes, but should work. If you want to check the file, you can use md5sum for example.
In alternative you can use a rar archive tool like Winrar. Open winrar, select all of files you want to transfer and split them to your target device with RAR type and split size of 32 GB.
In your target device copy all the files and unzip them.
This method is simple and works on any OS.
If it's Mac to Mac, AirDrop may be an option not available in general. Or you may be able to boot the Mac with the big file in Target Disk Mode, but that requires a cable you may not have, and the target machine requires a compatible port, unusual for non-Macs.
Depending on which version of OS X is on the Mac, you may already be running a file or web server, and if not, it may be easier to start one. So it can be as simple as temporarily moving the file in the right directory and using a web browser on the other computer to get it.
For any Unix-like systems, you can use
count to copy sections of the file to the USB stick, and then use
seek on the target box to stitch them back together. A large block size is used to make the math easier and maybe make the copy a little faster. This would not require extra space.
If the target is a Mac then something like:
source$ dd bs=1g if=bigfile.ext of=/Volumes/usb_stick/section count=30 skip=0 target$ dd bs=1g if=/Volumes/usb_stick/section of=newcopy.ext seek=0 source$ dd bs=1g if=bigfile.ext of=/Volumes/usb_stick/section count=30 skip=30 target$ dd bs=1g if=/Volumes/usb_stick/section of=newcopy.ext seek=30 source$ dd bs=1g if=bigfile.ext of=/Volumes/usb_stick/section count=30 skip=60 target$ dd bs=1g if=/Volumes/usb_stick/section of=newcopy.ext seek=60 # up to source$ dd bs=1g if=bigfile.ext of=/Volumes/usb_stick/section count=30 skip=180 target$ dd bs=1g if=/Volumes/usb_stick/section of=newcopy.ext seek=180
Since each section is 30GB, the USB stick must be formatted to allow files of that size: either HFS (Mac OS Extended) or ExFAT, but not FAT16.
If the target is Linux instead of Mac, its version of
dd uses uppercase for the block size multiplier:
bs=1G. Also, Linux does not support ExFAT, so worst case, you have to copy 4GB at a time with the USB stick formatted with FAT16.
If the target is Windows, you can boot a Linux live CD. Windows does support ExFAT, so could also copy each section off the USB stick into separate files, and then use
copy /b to stitch them back together. This would require double the space on the target.