2

I have a huge data set consisting of several files, with size of some of the files exceeding 30 GB. After compressing, transferring the data set over the network, and decompressing again I am suspecting something went wrong (that a file might be corrupt). One quick way to confirm this would be to compute checksums of both copies of a file---but there is a 2 GB limit on the size of the file for MD5 tool in Windows PowerShell.

Is there an alternative for computing a checksum of a file whose size exceeds 30 GB? (The tool should be implemented for both Linux and Windows.)

  • can you sshfs the windows machine from the linux machine or use the (I assume samba) network access to execute the md5 command on the file from the linux machine? – Fiximan Aug 14 '15 at 12:55
1

You can use openssl for that. It's available for Windows and Linux.

In a linux terminal use:

openssl md5 /path/to/your_file

In a windows command line:

openssl.exe md5 C:\path\to\your_file

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.