I have a Windows 10 Professional workstation I use to simulate material flow in impressions. The FEA software I use creates large 50-100GB database files for each simulation I run. Copying these files to spinning media for storage seems to not take advantage of the amount of RAM this system has while transferring, starting off quick for a second or two, and then dropping to the two RAID0 7200RPM disk's native transfer speed of 300 MB/s (171-342 seconds). The files are coming from a software RAID0 of two 600GB partitions on two 1.2TB Intel 750 PCIe NVMe SSDs, so read performance is not the issue. The system is on a 2200VA UPS with an EBM, and backed up nightly to our storage server, so no worries about data loss.
What I am wondering is:
If I can tweak Windows 10's cache settings to read the entirety of the 50-100GB files into RAM at the 4GB/s (12-25 seconds) the two Intel 750s are capable of, and then write them to disk transparently. I am under the impression that the built-in Windows feature "disk caching" is capable of this, but some default cache-size setting in Windows is limiting the cache to what looks like ~5GB (hence the small burst of speed at the start). I don't think this blip comes from the miserly 128MB cache on the destination drives, as "Modified" Physical Memory usage does go up by ~5GB in that first second or so of transfer. That 5GB can be seen being written to disk after the transfer dialog box disappears. RAM usage decreases in hand with the speed of the two 7200RPM drives in RAID0. Once completed, you can see the disk activity go to zero and the RAM utilization return to normal. This tells me that disk caching is at least working, just limited to 5GB at most.
The system would be fine using 50-100GB of it's available RAM for this transfer, as the simulations typically only use up to maybe 80GB of RAM, and the simulations are not using that amount of RAM until the last stages of the simulation.
I have a Dell Precision T7910 workstation with the following specs:
2P Xeon E5-2687W v4 256GB LRDIMM ECC DDR4 2400MHz Quadro M4000 x2 Intel 750 1.2TB PCIe NVMe, one boot, 600GB RAID0 Two Partitons x2 WD Gold 8TB in software RAID0 on SAS 12 GB Ports (128MB Cache) Eaton 5PX 2200 IRT 240V UPS Windows 10 Pro 1703 - System is old enough to not have Windows 10 Pro for Workstations
What I have tried:
Checked/Enabled: "Enable write caching on the device." - On each Disk Checked/Enabled: "Turn off Windows write-cache buffer flushing on the device." - On each Disk Made sure Superfetch service is running (for whatever good that does). Moved away from built-in hardware RAID, as there is NO cache anyway.
I have researched other topics with similar issues, and have come across another older thread mentioning a "CacheSet" tool:
Would this be applicable to my use-case or should I keep looking?
Is my understanding of how disk caching on a Windows platform works correct, or does it operate differently than I anticipated? I am just looking for write-caching to main memory, using maybe up to 100GB of RAM, nothing else.
Thank you for your help! Any suggestions are welcome.
EDIT: Running that cacheset.exe software as ADMIN reports the "Peak size" of "663732 KB" which seems too small (648MB?). Just not sure I want to commit changing this setting and potentially messing up this in-production system. The limit I keep running into is right around 5GB.
DOUBLE EDIT: I revised the apparent GBs that seem to be actually cached. The key was looking at "Modified Physical Memory" and seeing the ~5GB "cap" at the start of the transfer. Still looking to increase this to something like 100GB.
Thank you again!