Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top
sha1sum ./path/to/directory/* | sha1sum 

the above was posted as a way to compute a sha1sum of a directory which contains files. This command fails if the directory includes more directories. Is there a way to recursively compute the sha1sum of a directory of directories universally (without custom fitting an algorithm to the particular directory in question)?

share|improve this question
up vote 4 down vote accepted

Thanks to this SO post

find . -type f \( -exec sha1sum "$PWD"/{} \; \) | sha1sum

Warning: This code is untested! Edit this question if it's wrong and you can fix it; I'll approve your edit.

share|improve this answer
    
Sorry; I couldn't resist! ;-) Recursion is fun. Of course there's a way. I'll write up a proper answer now. – allquixotic Aug 6 '12 at 20:12

I generally like the "find | xargs" pattern, like so:

find ./path/to/directory/ -type f -print0  | xargs -0 sha1sum

You have to use the "-print0" and "-0", in case there are spaces in file names.

However, this is very similar to the "find -exec cmd {}" pattern.

See a discussion comparing the two patterns here: http://stackoverflow.com/questions/896808/find-exec-cmd-vs-xargs

share|improve this answer

This seems to work for me:

find . \( -not -name . \) -type f -exec cat {} + | sha1sum

EDIT: this will only sha1sum all of the files contained in the directory tree. If a directory's name was changed, this wouldn't catch it. Maybe something like:

find . -exec sha1sum {} + 2>&1 | sha1sum

Would do it. About the same answer as the other one though

share|improve this answer

I've observed that the order in which the find command outputs the found elements within a directory varies within identical directories on different partitions. If you're comparing the hashes of the same directory, you don't have to worry about that but if you're getting the hashes to ensure that no files were missed or corrupted during a copy, you need to include an additional line for sorting the content of the directory and it's elements. For example, Matthew Bohnsack's answer is quite elegant:

    find ./path/to/directory/ -type f -print0  | xargs -0 sha1sum

But if you're using it to compare a copied directory to it's original, you would send the output to a txt file which you would compare to the outputted list from the other directory using Kompare or WinMerge or by simply getting the hashes of each lis. The thing is, as the order in which the find tool will output the content may vary from one directory to another, Kompare will signal many differences because the hashes weren't calculted in the same order. Not a big deal for small directories but quite annoying if you're dealing with 30000 files. Therefore, you have do the extra steps of sorting the output to make it easier to compare the hash lists between the two directories.

    find ./path/to/directory/ -type f -print0  | xargs -0 sha1sum > sha1sum_list_unsorted.txt
    sort sha1sum_list_unsorted.txt > sha1sum_list_sorted.txt

This would sort the output so that files with same hash are going to be on the same lines when running the differencing program (provided that no files are missing the new directory).

Here's a script that I wrote. It does what the same thing that the find/xarg answer does but it will sort the files before getting the sha1sum (keeping them in the same directory). The first line of the script finds all the files within the directory recursively. The next one sorts the results alphabetically. The following two, takes the sorted content and appends a sha1sum and quotation marks to the files in the sorted list, making a big shell script that calculates each files hash, one at a time and outputs it to content_sha1sum.txt.

    find . -type f > content.txt
    sort content.txt > content_sorted.txt
    awk '{print "sha1sum \""$0}' content_sorted.txt > temp.txt
    awk '{print $0"\""}' temp.txt > get_sha1.sh
    chmod +x get_sha1.sh
    ./get_sha1.sh > content_sha1sum.txt
    rm content.txt
    rm content_sorted.txt
    rm temp.txt
    rm get_sha1.sh
    kate content_sha1sum.txt

Hope this helps.

share|improve this answer

Rather than have ONE enormous file containing all of the hashed information I was looking for a way to make a file in each folder of a tree. I took some inspiration from the comments here. Mine is a bit more complex than what is posted here. I use file rotation but this is the least complex for new players. This version will have it overwrite the old check sums with new ones. It may be good to keep 2-3 versions depending on how frequently you run it and your need for 'depth'.

[user@host bin]$ cat mkshaindir 
#!/bin/dash
cd $1
sha512sum * >.sha512sum

[user@host bin]$ find /var/tmp -type d -print0 | xargs -0 -i  mkshaindir  {}

Note that mkshaindir, for my purposes, is a separate component because there may be a need for me to make a hash of files in a new folder, or of one that was recently changed. This can all be combined in to one script if needed.

The rest is left as an exercise for the reader.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.