Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Under Windows 7 (64 bit), I'm reading through 9000 moderately sized files. In total, there is more than 200 MB of data. Using Java (JDK 1.6.21) I'm iterating over the files. The first 1400 or so go at full speed but then speed drops off to 4ms per file. It turns out that the main cost is incurred simply by opening the files.

I'm opening the files using new FileInputStream (and of course closing them in time to avoid file leaks). After some investigating, I see that Windows' disk cache is using only 100 MB or so of RAM although I have 8 GiB available. I've tried increasing the cache size using the CacheSet tool but any values I provide are considered out of range.

I've also tried enabling the LargeSystemCache registry key but (after rebooting) the CacheSet tool still indicates I'm using 100 MB of cache (and doesn't increase during the test run). Does anybody have any suggestions to "encourage" Windows 7 to cache my 9000 files?

share|improve this question

migrated from stackoverflow.com Aug 26 '10 at 9:09

This question came from our site for professional and enthusiast programmers.

1  
Where do you got the information from that only 100MB are used for disk cache? From what I know Windows uses by default all free memory for disk caching. A value of only 100MB seems to me like a minimum value. –  Robert Jul 18 '11 at 11:03
    
Part of the problem could be Java and it's JVM. There are tons of options to tweak the JVM - publib.boulder.ibm.com/infocenter/javasdk/v6r0/… . I think your best bet is to just write it in C++ or FreeBASIC (both which compile to native code and only limited by what the OS allows). –  Nathan Adams Feb 1 '13 at 20:05

4 Answers 4

I know this is old, but for others who run into this problem: You need to run CacheSet with administrative privileges manually. If you don't, you will get the OutOfRange error; it will NOT automatically ask for privileges.

share|improve this answer

I've tried increasing the cache size using the CacheSet tool but any values I provide are considered out of range.

  1. Make sure that you explicitly run CacheSet "As Administrator". CacheSet will say that the values are out of range when anything goes wrong.
  2. Make sure that the cache size is divisible by 4 KB

Also, I noticed that CacheSet has some trouble with large values, possibly because it is a 32-bit application from 2006. I went and wrote a 64-bit program around the same API, which allows setting cache sizes over 4GB: http://blog.thecybershadow.net/2012/12/14/64-bit-cacheset/

Note: The CacheSet article mentions CacheMan, however the latest version seems to use the same API as CacheSet - it's just packaged into an UI with a bunch of presets and other functionality.

share|improve this answer

Superfetch doesn't cache all of a large file, just the parts that are accessed. So accessing that file requires disk read still. A disk cache at 210mb like mine can cache everything from all those small files in Windows. And the performance is noticeable. I doubt though, that it would have impact on faster systems.

share|improve this answer

AnalogX CacheBooster is a free tool for Windows that lets you tweak the hard disk cache.

A discussion of this tool can be found in:
Increase Or Decrease Your Hard Drive Cache Memory With CacheBooster.

image

share|improve this answer
    
My current settings are all non-active and set to 0 Kb, does this mean this tool is not useful for me? I have 4 GB of RAM total and only 1.2 GB of RAM is used... –  Tom Wijsman Aug 26 '10 at 9:30
    
@TomWij: Where do you see that only 1.2 GB of RAM is used? Did you try to play with the CacheBooster values ? (create a system restore point first, just in case) –  harrymc Aug 26 '10 at 12:58
    
1.2 GB of ram is used, Process Explorer shows that. Just Windows memory... My System Restore is disabled, but anyhow I just want to know what this does before playing with it. Haven't tried that but I just wonder if it's normal that the sliders are all on 0 and active when I open CacheBooster? Does this mean caching is not enabled? Does this mean I don't need caching? ...? –  Tom Wijsman Aug 26 '10 at 23:05
    
@TomWij: Don't believe any such statistics, as nobody can say exactly how Windows uses memory. You may try playing with this utility to see if it helps, but take precautions as said above. –  harrymc Aug 27 '10 at 9:37
    
@TomWij: Whatever you are seeing is probably incorrect or doesn't apply. Windows 64-bits on 4GB RAM will use by default about 2GB for its own, mostly disk cache. Probably when 9000 files were read into memory, you filled out the cache memory and then things slowed down, because copying of files had to be finished for new files to be read into memory before copying. This utility can help you adjust the disk cache, but probably only DOWN, because Windows had already allocated the maximum possible. –  harrymc Aug 28 '10 at 14:33

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.