Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a huge /etc/hosts file with 18,000 lines. I use it to block adservers and such. My browsing performance seems to be fine, but I wonder if there's some size at which I'll start slowing myself down? How do OS's store the hosts file during operation? I can think of lots of ways, from slow to fast, and my a priori assumption is that it is handled efficiently since it is so central to networking. Here are some ways I can think of from simple to more complex:

  • Read from file each time it's needed.
  • Read once
    • linear search
    • sort, then binary search
    • use a trie
    • use a hash, but how does it minimize collisions

There's lots of ways that seem reasonable. What's actually used?

It seems like this question must have been answered somewhere, so pointers to an existing resource are welcome. I did look, but maybe not in the right place.

share|improve this question
To clarify, it's not the OS that handles /etc/hosts, but rather the libc. – Ignacio Vazquez-Abrams Oct 10 '10 at 7:48
To clarify further, libc, being the main interface between the kernel and the userland is considered part of the OS. – jlliagre Oct 10 '10 at 10:15
On Linux anyway, other operating systems may (and will) differ. – Cry Havok Oct 10 '10 at 10:36
Differ in what ? – jlliagre Oct 10 '10 at 13:57
So if I locate myself the source to libc somewhere, I should be able to find the answer? – user43031 Oct 11 '10 at 1:12

You should tell what OS you are using but anyway, a usual method to improve name resolution performance with Unix and Unix-like system is to use a daemon (commonly nscd) which caches positive and negative resolutions and quickly answers most requests. When /etc/hosts is modified, at least under Solaris, this cache is invalidated.

share|improve this answer
I use multiple variants of multiple operating systems. I would be interested in the details of how this is handled on any of them. The pointer to nscd is useful. I'm not worried about what happens when etc/hosts is modified, though. I am interested in how the cache is stored and accessed. – user43031 Oct 11 '10 at 0:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .