5

My system runs high on committed memory (out of 8GB RAM + 2 GB page file 85% memory is committed). Physical usage is at some 65%.

How can I identify what process(es) is allocating most of the committed memory? I understand that memory can be shared among processes. So far I've used VMMap to show committed memory but this is on a per-process base and doesn't take pagefile-backed sections into account.

enter image description here

enter image description here

2
  • So are you wanting to know what is using up all of your physical memory or what is using all of your virtual memory (physical + paged), or something else? Nov 5, 2018 at 13:10
  • I want to know what is responsible for the large commit charge on my system (the amount of virtual memory guaranteed to all processes).
    – Dunken
    Nov 5, 2018 at 17:11

4 Answers 4

10
+25

PowerShell Solution

1. Get top 10 processes using the highest amount of Virtual Memory

Get-Process | Sort PagedMemorySize-Desc | Select Name, PagedMemorySize, VirtualMemorySize -First 10

Output Example

Name                  VirtualMemorySize PagedMemorySize
----                  ----------------- ---------------
UrBackupClientBackend         685735936       548347904
explorer                     1529909248       478908416
Microsoft.Photos             1303465984       433094656
MBAMService                   661987328       228876288
MicrosoftEdgeCP               894496768       219799552
MsMpEng                       667783168       205774848
MicrosoftEdgeCP               874590208       202584064
mstsc                         440627200       185860096
javaw                         886177792       185556992
MicrosoftEdgeCP               802746368       146792448

2. Get sum of all committed Virtual Memory

Get-WmiObject win32_operatingsystem | Select @{L='commit';E={($_.totalvirtualmemorysize - $_.freevirtualmemory)*1KB/1GB}} 

Output Example

commit
------
4.56205749511719

Supporting Resources

4
  • 1
    @Dunken Run Get-Process | Select * -First 10 to see all properties you can get explicitly or whatever if needed. Nov 6, 2018 at 18:05
  • 1
    Process.VirtualMemorySize is the total defined v.a.s. of the process. It includes mapped and even reserved v.a.s. It is far larger than the process's contribution to commit charge. Nov 7, 2018 at 5:16
  • 1
    @JamieHanrahan Answer updated to make it more clear and with additional explanation.... Give that a shot with the Process.PrivateMemorySize Property added into the PowerShell command as mentioned. You can add PeakPagedMemorySize into the select statement to see the largest each processes virtual memory allocation has grown to for commit as well. Nov 7, 2018 at 5:51
  • 1
    Like pointed out by Jamie VirtualMemorySize doesn't seem to help here (far too large). Paged/PrivateMemorySize64 seems to be "Private Bytes" (or close) but is far smaller than what I get in VMMap as "Committed"... PeakPagedMemorySize64 is better but still far from what I see in VMMap: for example PPMS64 560MB vs. Committed Total 954MB
    – Dunken
    Nov 7, 2018 at 9:04
4

Process Explorer can show this information per-process :

image

Here is how to get the above screen in Process Explorer :

  • Click menu View > Show Lower Pane
  • Click menu View > Lower Pane View > DLLs
  • Click menu View > Show Unnamed Handles and Mappings
  • Click on a process in the upper pane
  • Right-click on the headers of the lower pane and choose Select Columns...
  • In the DLL tab, tick Mapped Size and Mappinq Type
  • Click OK

Process Hacker can similarly show this information, after choosing and double-clicking on a process, in the Handles tab uncheck Hide unnamed handles.

8
  • 1
    Maybe I miss something but if I sum up all the Size numbers shouldn't I get the same number as I can see in vmmap? Actually the sum is much lower value than what I would expect...
    – Dunken
    Nov 5, 2018 at 12:18
  • 1
    Which columns are you comparing?
    – harrymc
    Nov 5, 2018 at 12:27
  • In vmmap for example sqlservr.exe shows a total amount of committed memory of 1.6GB (1.3 Private Data). If I sum up the Sizes in ProcExp I get 230MB.
    – Dunken
    Nov 5, 2018 at 17:15
  • 1
    Yes, exactly. So how can I get the total? Add SUM(size) and Working Set?
    – Dunken
    Nov 6, 2018 at 7:26
  • 1
    Understood (and that's not an issue) but what would you take as a rough number?
    – Dunken
    Nov 6, 2018 at 7:38
1

In Process Explorer's processes list, the "Private Bytes" column shows each process's contribution to commit charge. It is not necessary to look at the lower pane view.

Be sure to run Process Explorer as Administrator.

Task Manager shows the same information on the Details tab in the "Commit size" column.

Note that what Task Manager shows in the "Memory (private working set)" column is not the same thing, even though it uses the word "private". That is showing the subset of each process's commit charge that happens to be in RAM for that process at the moment.

Per Windows Internals, the contributors to the total commit charge are:

  • private committed v.a.s. in each process
  • pagefile-backed mapped v.a.s. (does not show up in the process' "private bytes")
  • copy-on-write regions of mapped v.a.s.
  • Nonpaged and paged pool
  • other kernel-space allocations not explicitly backed by files (for example, pageable code in drivers or in ntoskrnl.exe does not count, as it is backed by the respective executable files)
  • Kernel stacks - every thread has one
  • Page tables
  • Space for page tables not yet actually allocated, but for which committed v.a.s. already exists
  • "Address Windowing Extension" (AWE) allocations

Windows Internals goes into more detail on what each of these things is and why each counts toward the systemwide commit charge. Unfortunately there are not counters for the virtual sizes of many of these things, which is what commit charge is about. RAMmap shows the physical sizes of a few of them but not the virtual.

12
  • 2
    Hmm... I don't think "Private Bytes" (in ProcExp) or "Commit size" (in TskMgr) shows the entire allocated, reserved memory. If I check in VMMap I get higher numbers. So I believe PrivateBytes/CommitSize is only part of the total...
    – Dunken
    Nov 6, 2018 at 7:30
  • You are correct, "Private Bytes" (in ProcExp) or "Commit size" (in TskMgr) does not include reserved virtual address space. But reserved v.a.s. is not committed memory and does not count toward commit charge. You asked how to find which processes contributed to committed memory - the counters I cited (really the same counter displayed with different names) are how you do that. (They do include pagefile-backed sections.) VMmap includes reserved v.a.s., and also v.a.s. mapped to files other than the pagefile. They don't contribute to commit charge, so you don't have to worry about them. Nov 6, 2018 at 9:20
  • 1
    I'm sorry, I am looking for committed memory: if I understand you right you're saying Sum(Private Bytes) is equal to total committed memory. However if I do that on my system "System Commit" (in ProcExp) is 15.4GB but Sum(Private Bytes) is 11.5 GB. Do I miss something?
    – Dunken
    Nov 6, 2018 at 11:05
  • No, I'm saying that each process's Private Bytes is its contribution to the total commit charge (= "system commit"). But there are others contributors to the latter total. The nonpaged pool, plus the virtual size of the paged pool, are usually the largest of the "system-wide" contributors. AWE allocations are another (because they take physical memory out of "circulation" ) but these are usually rare. Maybe you could post some screenshots? Nov 6, 2018 at 16:31
  • 1
    Please also add the "use counts" tab from RAMmap (sysinternals). Nov 6, 2018 at 17:37
0

So, from what I can gather, vmmap seems to be doing its own math (it does install a kernell mode driver to function properly) and not using the APIs in win32_process.

You can kind of recalculate what it is doing in the GUI from the CLI using the outfile <filename> arg which generates a .mmp file which is just XML.

Actually sleuthing contributions to commit size does not seem to be a trivial task as "sharable" can either be 1 process's problem or many depending. Private Data is most often where you'll have leaks, but you if you're just concerned with size you have to consider image, address window extensions, AWE, etc.

vmmap is a great tool but it does appear to be written for profiling the performance of a single process in mind.

The YouTube talk by Mark Russinovich (creator of sysinternals) is pretty instructive: Mysteries of Memory Management Revealed,with Mark Russinovich (Part 1). As others have mentioned, the Windows Internals book and Windows Troubleshooting books (also by Russinovich) are helpful too.

It is pretty frustrating this information isn't readily available within the OS.

example with notepad

This example will export a single snapshot from vmmap for notepad and read it back. It then adds up the Commit value from certain regions. This number seems to approximate what is shown the GUI.

vmmap_parse.ps1

Open a copy of notepad first. You can also open up vmmap and examine the process to compare.

function New-VmMapLogFile {
#Requires -RunAsAdministrator
[CmdletBinding()]
param(
  [Parameter(Mandatory=$True,ValueFromPipelineByPropertyName=$True)]
  [Alias('id')]
  [int32[]]
  $Ids,
  
  [Parameter(Mandatory=$False)]
  [ValidateNotNullOrEmpty()]
  [string]
  $DestinationPath = (Join-Path $PWD.Path "vmmap_$(get-date -f 'yyyyMMdd_HHmmss')"),
  
  [Parameter(Mandatory=$False)]
  [ValidateNotNullOrEmpty()]
  [string]
  $VmMapPath = 'c:\sysinternals\vmmap.exe'
)
BEGIN {
  if(-not (Test-Path -Path $DestinationPath)) {
    New-Item -Path $DestinationPath -ItemType 'container' -ea 1 | out-null
  }
  if(-not (Test-Path -Path $VmMapPath -PathType 'leaf')) {
    throw "vmmap missing"
  }
}
PROCESS {
  foreach($id in $Ids) {
    $proc = $null
    $proc = Get-Process -Id $id -ea 0
    if(-not $proc) {
      write-warning "cannot find PID $($id)"
      continue
    }
    [System.IO.FileInfo]$outputfile = Join-Path $DestinationPath "$($id).mmp"
    write-verbose "creating $($outputfile.FullName)"
    $vmmapProcArgs = @{
        Wait = $True
        FilePath = $VmMapPath
        ArgumentList = @(
            '-accepteula',
            '-p',$id,
            'outputfile',$outputfile.FullName
        )
        WindowStyle = 'hidden'
    }
    Start-Process @vmmapProcArgs
    if(@(0) -notcontains $LASTEXITCODE) {
      write-warning "vmmap PID $($id) returned: $($LASTEXITCODE)"
    }
  }
}}

function Import-VMMapLogFiles {
param(
  [Parameter(Mandatory=$True)]
  [ValidateNotNullOrEmpty()]
  [ValidateScript({ Test-Path -Path $_ -PathType 'container' })]
  [string]
  $DirectoryPath
)
    write-verbose "Checking dir $($DirectoryPath) for *.mmp files"
    $mmpFiles = @(Get-ChildItem -Path $DirectoryPath -File -Filter '*.mmp')
    write-verbose "$($mmpFiles.Count) files found"
    foreach($mmpFile in $mmpFiles) {
        $objProps = [ordered]@{
            PID = $vmmap.root.PID
            Process = $vmmap.root.Process
        }
        # read XML file
        [xml]$vmmap = Get-Content -Path $mmpFile.FullName -Raw
        $regions = @($vmmap.root.Snapshots.Snapshot.MemoryRegions.Region)
        $regionByType = $regions | Group-Object -Property 'Type'
        
        # examine regions
        $totalCommitKb = 0
        foreach($r in $regionByType) {
            $keyPrefix = ($r.Name.ToLower() -replace '\s+','_')

            # filter regions
            $validRegions = @()
            switch($r.Name) {
                'Private Data' {
                    # 4096 regions only
                    $validRegions = @($R.Group | Where-Object { $_.Size - 4096 })
                }
                default {
                    $validRegions = @($R.Group)
                }
            }

            # commited sum
            $commitKeyRegex = 'image|shareable|private_data|heap|stack'
            $commitRegions = @($validRegions | where-object { $_.Commit -gt 0 })
            $commitBytes = ($commitRegions | Select-Object -ExpandProperty Commit | Measure-Object -Sum | Select-Object -ExpandProperty Sum)
            $commitKb = $commitBytes / 1KB
            $commitRounded = [math]::Round($commitKb, 2)
            if($keyPrefix -match $commitKeyRegex) {
                $totalCommitKb += $commitRounded
            }

            # size sum
            $sizeBytes = 0
            $sizeBytes = $validRegions | Select-Object -ExpandProperty Size | Measure-Object -Sum | Select-Object -ExpandProperty Sum
            $sizeKb = $sizeBytes / 1KB
            $sizeKbRounded = [math]::Round($sizeKb, 2)
            
            # add properties
            $objProps["$($keyPrefix)_kb"] = $sizeKbRounded
            $objProps["$($keyPrefix)_commit_kb"] = $commitRounded
        }
        $objProps['commit_total_kb'] = $totalCommitKb
        [pscustomobject]$objProps
    }
}

$tmpDir = Join-Path $PSScriptRoot 'notepad'
Remove-Item -Path $tmpDir -recurse -force -ea 0 | out-null

get-process -name 'notepad' -ea 1 | New-VmMapLogFile -DestinationPath $tmpDir
Import-VMMapLogFiles -Verbose -DirectoryPath $tmpDir

results

from gui vmmap GUI

script output

PID                           : 15320
Process                       : "C:\Windows\System32\notepad.exe"
free_kb                       : 135287120640
free_commit_kb                : 0
private_data_kb               : 4238760
private_data_commit_kb        : 316
unusable_kb                   : 2904
unusable_commit_kb            : 0
thread_stack_kb               : 512
thread_stack_commit_kb        : 80
heap_(shareable)_kb           : 64
heap_(shareable)_commit_kb    : 64
shareable_kb                  : 2147510224
shareable_commit_kb           : 46412
mapped_file_kb                : 23040
mapped_file_commit_kb         : 23040
heap_(private_data)_kb        : 4516
heap_(private_data)_commit_kb : 636
image_(aslr)_kb               : 52720
image_(aslr)_commit_kb        : 52720
commit_total_kb               : 123268

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .