Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

In the Dash of my Ubuntu/Linux system are two versions of the same program.

Initial Problem

To find where the corresponding .desktop files are located I used

find / -type f -name 'Sublime Text.desktop' 2> /dev/null

I got zero hits, so I did (with success)

find / -type f -name '[s,S]ublime*.desktop' 2> /dev/null

I was amazed as I saw, that it finished after about three seconds, as the search term should be significant lager than the first one. As it was not quiet kosher to me, I ran the first command again and to my surprise now it took only about three seconds to finish too.

To verify the behaviour I powered up a second Linux box and ran the first command again, but this time with time

time find -type f -name 'Sublime Text.desktop' 2> /dev/null 

Proof


find does not only speed up the search of the same search term, but rather all searches (within the same path?). Even the search for an "unreleated" String is not slowed down.

time find / -type f -name 'Emilbus Txet.Potksed' 2> /dev/null

Analyse RAM while before and after using find

What does find do to speed the search process up so insanely?

share|improve this question
    
(from an outsider perspective) maybe find indexes the results it gets, so next executions are run faster. –  Doktoro Reichard Aug 30 '13 at 21:16
    
Would it be possible to index all files it finds, even those that do not match the search term? Wouldn't that exceed storage capabilities of the RAM? And if not why is find not indexing everything by default? If it indexes the matches only, then my last example with the reversed search term should not complete that quick. –  Creaturo Aug 30 '13 at 21:30

1 Answer 1

up vote 4 down vote accepted

The reason that find is faster on the second time is that linux does file caching. Whenever a file is accessed the first time, it keeps the contents of the file in memory (of course it only does that when you have free RAM available). If the file is read again at a later time, it can then just fetch the contents from memory without actually having to read the file again. Because memory access is much faster than disk access, this increases overall performace.

So what happens is that on the first find, most of the files aren't in memory yet, so linux has to do lots of disk operations. This is slow, so it takes some time.

When you executefind again, most of the files and directories are already in memory and it's much faster.


You can test this out yourself if you clear the cache between the two find executions. Then the second find won't be faster than the first one. Here is how it looks on my system:

# This clears the cache. Be careful through, you might loose some data (Although this shouldn't happen, it's better to be sure)
$ sync && echo 3 | sudo tee /proc/sys/vm/drop_caches
3

$ time find /usr/lib -name "lib*"
find /usr/lib/ -name "lib*"  0,47s user 1,41s system 8% cpu 21,435 total

# Now the file names are in the cache. The next find is very fast:
$ time find /usr/lib -name "lib*"
find /usr/lib/ -name "lib*"  0,19s user 0,28s system 69% cpu 0,673 total

# If we clear the cache, the time goes back to the starting time again
$ sync && echo 3 | sudo tee /proc/sys/vm/drop_caches
3
$ time find /usr/lib -name "lib*"
find /usr/lib/ -name "lib*"  0,39s user 1,45s system 10% cpu 16,866 total
share|improve this answer
    
Amazing, does find cache every file/directory/link in /usr/lib? If so, find / -name 'foo.bar' would cache the whole file system, right? As the caching is only using a small amount of RAM, is there a reason, why the OS does not cache the entire file system by default, as soon as the HDD/SSD is idle? PS: Could you add a comment to the second line of your script? I do not get why you are writing 3 to /proc/sys/vm/drop_caches. –  Creaturo Aug 30 '13 at 21:55
1  
@th3m3s I guess that find / ... only caches the file names / directory contents. I think it doesn't cache the entire filesystem by default to have to cache free for other things, for example files you read. –  bennofs Aug 30 '13 at 21:58
1  
Correct, find does not access the contents of the files, only the directory structure and file names, so it only caches that part. In general practice, every time a file is read via the relevant kernel call, Linux will first check RAM to see if it's there, and if not: actually read it from disk (and normally store it in RAM for future lookup). One can directly see that this extra lookup must come with a certain performance penalty if cached files are never found, but the substantial "profit" when you have a cache hit by far outweighs this. –  Daniel Andersson Aug 30 '13 at 22:05

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.