30

This question already has an answer here:

I have noticed this on many different Windows machine, on many different occasions: the RAM usage reported by Task Manager or Resource Monitor often seems to add up to an amount which is significantly lower than the actual amount in use.

For example, many times on my laptop or my desktop, I have seen something like 7GB in use, and yet the total of the Working RAM Set is more like 3GB. I just can't figure out where it's being used!

Here's an extreme example I noticed today in Resource Monitor on a server:

Resource monitor
Click for full size

If you right-click the image and open in a new tab, and view the numbers, you'll notice that the Working Set (which doesn't include nonphysical virtual memory) add up to about 1.7GB. I get similar numbers by adding up RAM usage in Task Manager when "Show processes from all users" is enabled.

Now here's a screenshot of task manager's Performance tab:

Task manager
Click for full size

This says 7.6GB of physical memory is in use.

I see this all the time, on personal computers, laptops, and now servers: the total RAM usage reported by system tools only accounts for about 1/4 of the RAM usage I observe. WTF is going on???

Is there any satisfying explanation of where all my RAM is? What is gobbling it up, and why is it leaving no trace?

EDIT: Here's a picture of the graphical RAM use, as user whs asked for:

RamMap Use
Click for full size

EDIT 2: In response to James' response, here's a picture of the nonpaged processes in poolmon.exe, sorted by size:

enter image description here

These results confuse me. poolmon correctly states that I have 6GB of nonpaged pool in use, but all the nonpaged pool processes are less than 8MB in size.

What could this mean? Is poolmon failing to detect some of the processes using the nonpaged pool?

marked as duplicate by magicandre1981, fixer1234, Ramhound, bwDraco, Art Gertner Apr 27 '15 at 10:14

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • Can you please post a picture of your Resource Monitor > Memory tab (just the bottom part with the colors and numbers). That would be more telling. – whs Apr 25 '15 at 22:31
  • 2
    @Ramhound: Thank you for pointing me to that link! Unfortunately, as whs just requested, I've uploaded a picture of Resource Monitor's Physical Memory graphical map, and the Standby portion of the Physical Memory is less than 2% of the total RAM use. David Schwartz's answer suggests that "hidden" RAM use comes from Standby RAM (ie, RAM used for cached data and code not in use), and that is not the case here. As such, I don't think David's answer explains what is going on. – DumpsterDoofus Apr 25 '15 at 22:58
  • Use the program RamMap, and explore with it more., So far it might be some driver going bonkers? but indeed it is not the standby (cache). Do you run any special programs that alledge to fix up the ram, or that suppose to speed up the computer? Any odd driver items that would be special to your computers? – Psycogeek Apr 26 '15 at 3:29
  • 2
    The link to the "answered before" does not cover this case. – Jamie Hanrahan Apr 26 '15 at 3:56
29

I'm sorry, I know this sounds like a flippant response... but the answer to the question in your title is "because they're not supposed to."

Or to put it more politely: There is much use of RAM that is not in processes' private working sets. Some of it is in processes' shared working sets - but you can't get a reliable notion of the actual usage there, because of sharing; adding up the processes' numbers will give you far too large a result.

Other stuff that occupies RAM, like the nonpaged pool, the resident portion of the paged pool, and the resident portions of other kernel-space uses, doesn't show up in Task Manager's "processes" display at all.

Regarding your specific issue:

On the Task manager display, see the "kernel memory" section? You have 6 GB of "nonpaged memory" (that's nonpaged pool). That's part of the "In use" section in your second graph. The nonpaged pool is not charged to any process, which is why adding up the per-process numbers in task manager doesn't get close to the total in use. Some driver is most likely using it. This is a wholly excessive amount; it should be well under 1 GB. whatever driver is responsible for the excessive part of the nonpaged pool usage is unquestionably buggy.

RAMmap can confirm this (on its "Use Counts" tab, look at the total for "Nonpaged Pool") but it can't help you find which driver is causing it.

Here's how to find it: Obtain a copy of the Microsoft tool "poolmon". It is a character-mode tool (boy, is it ever) distributed with the Windows Driver Kit. For Windows 7 the WDK is a free download. You have to download the whole thing (it's an ISO) and install it from that, but you can choose to install just the tools, if that's all you want.

Find poolmon in the WDK directories - be sure to pick the right one, 32- or 64-bit - and run it from an administrator command prompt. You will get a display like this:

enter image description here

Now, press the "p" key (no, I'm not kidding. No menus here!) until the "Type" column shows only "Nonp". Then press "b" (twice if necessary) to sort the display in descending order by the Bytes column (that was already done in the sample here).

Then look at the "Tag" column for the topmost line. In the (obviously artificial) case shown here it's "Leak". (This system is running a driver that was deliberately bugged to cause this problem- it's "leaking" nonpaged pool.)

btw, the highlighted lines are the ones that changed since the previous update to this archaic screen.

Now search c:\Windows\System32\Drivers for a .sys file containing that string. In this case you'd be looking for "Leak", like this:

c:\windows\system32> findstr /s Leak *.sys

Then search the web for references to that string and/or that driver name.

Returning here and reporting the full name, manufacturer name, etc. from the .sys file would be helpful too.

(My bet is that the tag you find will be ECMC, the driver is intmsd.sys, and it's associated with a product called ExpressCache or IntelliMemory. I would "uninstall" that product. There is an update to fix the problem, but even with the fixed version I have never seen a system's performance improved by this product; it essentially duplicates functionality that's already in Windows.)

If you can't find it that way, the next step is to use the "Windows Performance Toolkit". Search this forum for that string, with answers by magicandre1981, for a how-to. Ignore answers that mention xperf - it is an older version of the tool.

UPDATE: Per the comments, the OP did the above and found that though poolmon reported the total size of the nonpaged pool was indeed huge, all of the allocated pieces were apparently tiny. My conjecture (also in the comments) is that this is due to what I will call "bloated" pool: Pool was allocated, then freed, but for some reason that amount of RAM allocated to pool was not shrunk to reflect the "freeing". Following the procedure described in this answer by magicandre may identify the culprit.

  • Thank you for taking to investigate further than others have! I will poke around for a leak using poolmon today if I have time, and try to read your answer more closely as well to make sure I understand it. Are you saying there is probably a memory leak? If so, that's entirely plausible, as we're working on software known to have some horrible memory leaks. – DumpsterDoofus Apr 27 '15 at 15:37
  • @DumpsterDoofus: Yes. There is probably a memory leak in a kernel mode driver. This is different from a memory leak in an app. – Jamie Hanrahan Apr 27 '15 at 16:27
  • So I finally got around to running poolmon.exe on the server in question, and edited my question to include a screenshot. poolmon states I have 6GB nonpaged pool (as you pointed out from my Task Manager screenshot), but when I look at only "Nonp" processes and sort by size, they're all tiny (the biggest is 8MB). Do you have any ideas why poolmon isn't detecting the bulk of the nonpaged pool usage? – DumpsterDoofus May 7 '15 at 22:46
  • 2
    Ah... the problem here is in the word "usage". So you have 6 GB nonpaged pool, but it appears that only a fraction of it is in use at the moment. Alas nothing I know of (not TM, not poolmon, not RAMmap) shows how much of the pool is actually in use. My guess is that there was a heavy pool user at one time, so the pool was expanded to accommodate it, and then the heavy user went away. Due to the way pool is handled it is not easy to free the RAM once allocated to it, unless the allocation was in large contiguous chunks. I would suggest watching this after a fresh system startup. – Jamie Hanrahan May 7 '15 at 23:55
  • There's a version of poolmon.exe available in the Support Tools for Server 2003 (serverfault.com/questions/84479/…) - a bit old, but it's a mere 5.2MB download, and still runs on (at least) Server 2016. – mwfearnley Apr 23 at 9:56

Not the answer you're looking for? Browse other questions tagged or ask your own question.