6

I'd like to know how to create a list with md5sums from files in the current directory - files that are over a specified size. I can do one or the other, but I don't know how to combine the two.

8

You can pipe find (which will list all files you are interested in) output to md5sum.

xargs is needed to avoid creating a loop.

To write it up in a command:

find . -maxdepth 1 -size +30M -type f -print0 | xargs -0 md5sum

  • . says "start listing items from current directory"
  • -maxdepth 1 specifies to list only files in this directory (do not descend deeper)
  • -size +30M specifies to list only files which exceed 30 megabytes of space (you can also use k and G suffixes if needed, read more about its possibilities in man find if needed)
  • -type f avoids listing directories - you can't count md5sum for a directory
  • -print0 makes find separate filenames with the null byte. We use this because everything fails when you put a newline in filename.

xargs will take stuff from standard input (thanks to -0 flag it will treat the null byte as record separator) and feed them as arguments to md5sum.

You can also make it without the pipe, but I find that syntax confusing and I prefer to pipe it to xargs: find . -maxdepth 1 -size +30M -type f -execdir md5sum {} \;

And, as @David writes in the comments, you can put + after the {}, i.e. find . -maxdepth 1 -size +30M -type f -execdir md5sum {} + \;.

What does it change? md5sum can be called for two files in two ways: md5sum file1; md5sum file2 or md5sum file1 file2. Without the + you get the first option, adding + results in executing it the second way. The most important benefit is speed, since md5sum executes only once. It may not be that beneficial for some programs, but for some cases, for example a program can then run on many cores and speed-up the work by the factor of NUM_CPUS.

About that strange syntax (from man find):

-execdir command ;

Execute command; true if 0 status is returned. All following arguments to find are taken to be arguments to the command until an argument consisting of ';' is encountered. The string '{}' is replaced by the current file name being processed everywhere it occurs in the arguments to the command, not just in arguments where it is alone, as in some versions of find. Both of these constructions might need to be escaped (with a '\') or quoted to protect them from expansion by the shell. See the EXAMPLES section for examples of the use of the -execdir option. The specified command is run once for each matched file.

  • Very informative I will try this ! How do I get this into a text file ? > checksums.md5 ? – Bug J. Jun 23 '17 at 0:34
  • Sure, just put > checksums.md5 at the end of this "script". Just take in mind, that if you put checksums.md5 in your current directory it will have its (the file's checksums.md5) hash in it and this hash will be incorrect... It's a hard problem to create a file which has its own hash inside. – styrofoam fly Jun 23 '17 at 0:36
  • 3
    If you have GNU find you can use -exec md5sum {} + to pass all matches to a single md5sum command (assuming they all fit onto a single command-line). – David Foerster Jun 23 '17 at 7:13

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.