If you you modify a previously clean file, git notices. It even does so if you set the modified data of that file back what what it previously was. Yet git doesn't read the data of all tracked files in the working directory. It can't do because that'd be way too fast. I have repos many GB in size and git status is super fast, way to fast to have read all of the data in the repo.

  • How does git do this?
  • How can one make git check the contents of every tracked file to verify they haven't been corrupted since they last have been committed? Preferably without checking the contents of the repo out in a different location and asking a diff tool about possible differences in the working directories.

To better understand what can't be going on (rule out a few options), I conducted this experiment: I created a file system inside a file and copied my documents repo (3 GB of tracked files) there. Then, I unmounted it, opened the file the file system is stored in in a hex editor, searched for the word "regardless" and swapped an occurrence of it out for the word "relentless" (equal amount of storage space needed so I don't mess the layout up).

I then mounted the file system again and sure enough, git didn't notice something changed. I made sure the file I made the change in is tracked by opening it and searching for the word "relentless". Actually, the first time I swapped the word in a backup file (*~ file) out and I don't track those. But the second time the change occurred in a tracked file.

Even after I viewed the file (so their access time stamp is changed), git didn't notice the change. If this was a corruption because a cosmic ray hit my hard drive and changed the word "regardless" in this occurrence for the word "relentless", how would I make git notice this for me?

  • “It can't do because that'd be way too fast.” Sometimes. I have done git status on relatively small repositories and the process hangs. – JakeGould Jan 4 '17 at 0:10
  • I have done git status on repos > 50 GB stored on external hard drives connected via USB and the command returned within a split second. I do git status daily on my documents repo which currently holds 3 GB of tracked data in the working tree. Granted, I use an SSD but the SSD reads with less than 600 MB/s and the command definitely doesn't take 5 seconds to return. It returns in a split second. Instantly, for a human, though time measures it as follows: real 0m0.032s; user 0m0.008s; sys 0m0.028s (I exchanged the line breaks for semicolons.) Way below 6 s, though. – UTF-8 Jan 4 '17 at 0:20
  • If Git interacts with the filesystem, it can quickly detect what files are marked as “changed” or “modified” or even “new” via inodes and then do a quick MD5 check on the files to see of the contents are changed. Granted, this doesn't completely factor in large files but it seems like that might be the case here. – JakeGould Jan 4 '17 at 0:25
  • @JakeGould Git doesn't use md5. It uses sha1. – UTF-8 Feb 21 '17 at 22:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.