Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Under Windows 7 (64 bit), I'm reading through 9000 moderately sized files. In total, there is more than 200 MB of data. Using Java (JDK 1.6.21) I'm iterating over the files. The first 1400 or so go at full speed but then speed drops off to 4ms per file. It turns out that the main cost is incurred simply by opening the files.

I'm opening the files using new FileInputStream (and of course closing them in time to avoid file leaks). After some investigating, I see that Windows' disk cache is using only 100 MB or so of RAM although I have 8 GiB available. I've tried increasing the cache size using the CacheSet tool but any values I provide are considered out of range.

I've also tried enabling the LargeSystemCache registry key but (after rebooting) the CacheSet tool still indicates I'm using 100 MB of cache (and doesn't increase during the test run). Does anybody have any suggestions to "encourage" Windows 7 to cache my 9000 files?

share|improve this question

migrated from stackoverflow.com Aug 26 '10 at 9:09

This question came from our site for professional and enthusiast programmers.

3  
Where do you got the information from that only 100MB are used for disk cache? From what I know Windows uses by default all free memory for disk caching. A value of only 100MB seems to me like a minimum value. – Robert Jul 18 '11 at 11:03
    
Part of the problem could be Java and it's JVM. There are tons of options to tweak the JVM - publib.boulder.ibm.com/infocenter/javasdk/v6r0/… . I think your best bet is to just write it in C++ or FreeBASIC (both which compile to native code and only limited by what the OS allows). – Nathan Adams Feb 1 '13 at 20:05

Superfetch doesn't cache all of a large file, just the parts that are accessed. So accessing that file requires disk read still. A disk cache at 210mb like mine can cache everything from all those small files in Windows. And the performance is noticeable. I doubt though, that it would have impact on faster systems.

share|improve this answer

I've tried increasing the cache size using the CacheSet tool but any values I provide are considered out of range.

  1. Make sure that you explicitly run CacheSet "As Administrator". CacheSet will say that the values are out of range when anything goes wrong.
  2. Make sure that the cache size is divisible by 4 KB

Also, I noticed that CacheSet has some trouble with large values, possibly because it is a 32-bit application from 2006. I went and wrote a 64-bit program around the same API, which allows setting cache sizes over 4GB: http://blog.thecybershadow.net/2012/12/14/64-bit-cacheset/

Note: The CacheSet article mentions CacheMan, however the latest version seems to use the same API as CacheSet - it's just packaged into an UI with a bunch of presets and other functionality.

share|improve this answer

I know this is old, but for others who run into this problem: You need to run CacheSet with administrative privileges manually. If you don't, you will get the OutOfRange error; it will NOT automatically ask for privileges.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.