How to identify which process committed memory

Question

My system runs high on committed memory (out of 8GB RAM + 2 GB page file 85% memory is committed). Physical usage is at some 65%.

How can I identify what process(es) is allocating most of the committed memory? I understand that memory can be shared among processes. So far I've used VMMap to show committed memory but this is on a per-process base and doesn't take pagefile-backed sections into account.

Answer 1

PowerShell Solution

1. Get top 10 processes using the highest amount of Virtual Memory

Get-Process | Sort PagedMemorySize-Desc | Select Name, PagedMemorySize, VirtualMemorySize -First 10

Output Example

Name                  VirtualMemorySize PagedMemorySize
----                  ----------------- ---------------
UrBackupClientBackend         685735936       548347904
explorer                     1529909248       478908416
Microsoft.Photos             1303465984       433094656
MBAMService                   661987328       228876288
MicrosoftEdgeCP               894496768       219799552
MsMpEng                       667783168       205774848
MicrosoftEdgeCP               874590208       202584064
mstsc                         440627200       185860096
javaw                         886177792       185556992
MicrosoftEdgeCP               802746368       146792448

2. Get sum of all committed Virtual Memory

Get-WmiObject win32_operatingsystem | Select @{L='commit';E={($_.totalvirtualmemorysize - $_.freevirtualmemory)*1KB/1GB}} 

Output Example

commit
------
4.56205749511719

---

Supporting Resources

• How to get Memory Committed Bytes per process

• Process.VirtualMemorySize Property

  "The amount of virtual memory, in bytes, that the associated process has requested."

• Use PowerShell to Find System Committed Memory

• Win32_OperatingSystem class

  TotalVirtualMemorySize

  Data type: uint64

  Access type: Read-only

  Qualifiers: Units ("kilobytes")

  Number, in kilobytes, of virtual memory. For example, this may be calculated by adding the amount of total RAM to the amount of paging space, that is, adding the amount of memory in or aggregated by the computer system to the property, SizeStoredInPagingFiles.

• Process.PeakPagedMemorySize Property

• Process.PagedMemorySize Property

  "Gets the amount of paged memory, in bytes, allocated for the associated process."

  "The amount of memory, in bytes, allocated by the associated process that can be written to the virtual memory paging file."

• Process.PrivateMemorySize Property

Answer 2

Process Explorer can show this information per-process :

Here is how to get the above screen in Process Explorer :

• Click menu View > Show Lower Pane
• Click menu View > Lower Pane View > DLLs
• Click menu View > Show Unnamed Handles and Mappings
• Click on a process in the upper pane
• Right-click on the headers of the lower pane and choose Select Columns...
• In the DLL tab, tick Mapped Size and Mappinq Type
• Click OK

Process Hacker can similarly show this information, after choosing and double-clicking on a process, in the Handles tab uncheck Hide unnamed handles.

Answer 3

In Process Explorer's processes list, the "Private Bytes" column shows each process's contribution to commit charge. It is not necessary to look at the lower pane view.

Be sure to run Process Explorer as Administrator.

Task Manager shows the same information on the Details tab in the "Commit size" column.

Note that what Task Manager shows in the "Memory (private working set)" column is not the same thing, even though it uses the word "private". That is showing the subset of each process's commit charge that happens to be in RAM for that process at the moment.

Per Windows Internals, the contributors to the total commit charge are:

• private committed v.a.s. in each process
• pagefile-backed mapped v.a.s. (does not show up in the process' "private bytes")
• copy-on-write regions of mapped v.a.s.
• Nonpaged and paged pool
• other kernel-space allocations not explicitly backed by files (for example, pageable code in drivers or in ntoskrnl.exe does not count, as it is backed by the respective executable files)
• Kernel stacks - every thread has one
• Page tables
• Space for page tables not yet actually allocated, but for which committed v.a.s. already exists
• "Address Windowing Extension" (AWE) allocations

Windows Internals goes into more detail on what each of these things is and why each counts toward the systemwide commit charge. Unfortunately there are not counters for the virtual sizes of many of these things, which is what commit charge is about. RAMmap shows the physical sizes of a few of them but not the virtual.

Answer 4

So, from what I can gather, vmmap seems to be doing its own math (it does install a kernell mode driver to function properly) and not using the APIs in win32_process.

You can kind of recalculate what it is doing in the GUI from the CLI using the outfile <filename> arg which generates a .mmp file which is just XML.

Actually sleuthing contributions to commit size does not seem to be a trivial task as "sharable" can either be 1 process's problem or many depending. Private Data is most often where you'll have leaks, but you if you're just concerned with size you have to consider image, address window extensions, AWE, etc.

vmmap is a great tool but it does appear to be written for profiling the performance of a single process in mind.

The YouTube talk by Mark Russinovich (creator of sysinternals) is pretty instructive: Mysteries of Memory Management Revealed,with Mark Russinovich (Part 1). As others have mentioned, the Windows Internals book and Windows Troubleshooting books (also by Russinovich) are helpful too.

It is pretty frustrating this information isn't readily available within the OS.

example with notepad

This example will export a single snapshot from vmmap for notepad and read it back. It then adds up the Commit value from certain regions. This number seems to approximate what is shown the GUI.

vmmap_parse.ps1

Open a copy of notepad first. You can also open up vmmap and examine the process to compare.

function New-VmMapLogFile {
#Requires -RunAsAdministrator
[CmdletBinding()]
param(
  [Parameter(Mandatory=$True,ValueFromPipelineByPropertyName=$True)]
  [Alias('id')]
  [int32[]]
  $Ids,
  
  [Parameter(Mandatory=$False)]
  [ValidateNotNullOrEmpty()]
  [string]
  $DestinationPath = (Join-Path $PWD.Path "vmmap_$(get-date -f 'yyyyMMdd_HHmmss')"),
  
  [Parameter(Mandatory=$False)]
  [ValidateNotNullOrEmpty()]
  [string]
  $VmMapPath = 'c:\sysinternals\vmmap.exe'
)
BEGIN {
  if(-not (Test-Path -Path $DestinationPath)) {
    New-Item -Path $DestinationPath -ItemType 'container' -ea 1 | out-null
  }
  if(-not (Test-Path -Path $VmMapPath -PathType 'leaf')) {
    throw "vmmap missing"
  }
}
PROCESS {
  foreach($id in $Ids) {
    $proc = $null
    $proc = Get-Process -Id $id -ea 0
    if(-not $proc) {
      write-warning "cannot find PID $($id)"
      continue
    }
    [System.IO.FileInfo]$outputfile = Join-Path $DestinationPath "$($id).mmp"
    write-verbose "creating $($outputfile.FullName)"
    $vmmapProcArgs = @{
        Wait = $True
        FilePath = $VmMapPath
        ArgumentList = @(
            '-accepteula',
            '-p',$id,
            'outputfile',$outputfile.FullName
        )
        WindowStyle = 'hidden'
    }
    Start-Process @vmmapProcArgs
    if(@(0) -notcontains $LASTEXITCODE) {
      write-warning "vmmap PID $($id) returned: $($LASTEXITCODE)"
    }
  }
}}

function Import-VMMapLogFiles {
param(
  [Parameter(Mandatory=$True)]
  [ValidateNotNullOrEmpty()]
  [ValidateScript({ Test-Path -Path $_ -PathType 'container' })]
  [string]
  $DirectoryPath
)
    write-verbose "Checking dir $($DirectoryPath) for *.mmp files"
    $mmpFiles = @(Get-ChildItem -Path $DirectoryPath -File -Filter '*.mmp')
    write-verbose "$($mmpFiles.Count) files found"
    foreach($mmpFile in $mmpFiles) {
        $objProps = [ordered]@{
            PID = $vmmap.root.PID
            Process = $vmmap.root.Process
        }
        # read XML file
        [xml]$vmmap = Get-Content -Path $mmpFile.FullName -Raw
        $regions = @($vmmap.root.Snapshots.Snapshot.MemoryRegions.Region)
        $regionByType = $regions | Group-Object -Property 'Type'
        
        # examine regions
        $totalCommitKb = 0
        foreach($r in $regionByType) {
            $keyPrefix = ($r.Name.ToLower() -replace '\s+','_')

            # filter regions
            $validRegions = @()
            switch($r.Name) {
                'Private Data' {
                    # 4096 regions only
                    $validRegions = @($R.Group | Where-Object { $_.Size - 4096 })
                }
                default {
                    $validRegions = @($R.Group)
                }
            }

            # commited sum
            $commitKeyRegex = 'image|shareable|private_data|heap|stack'
            $commitRegions = @($validRegions | where-object { $_.Commit -gt 0 })
            $commitBytes = ($commitRegions | Select-Object -ExpandProperty Commit | Measure-Object -Sum | Select-Object -ExpandProperty Sum)
            $commitKb = $commitBytes / 1KB
            $commitRounded = [math]::Round($commitKb, 2)
            if($keyPrefix -match $commitKeyRegex) {
                $totalCommitKb += $commitRounded
            }

            # size sum
            $sizeBytes = 0
            $sizeBytes = $validRegions | Select-Object -ExpandProperty Size | Measure-Object -Sum | Select-Object -ExpandProperty Sum
            $sizeKb = $sizeBytes / 1KB
            $sizeKbRounded = [math]::Round($sizeKb, 2)
            
            # add properties
            $objProps["$($keyPrefix)_kb"] = $sizeKbRounded
            $objProps["$($keyPrefix)_commit_kb"] = $commitRounded
        }
        $objProps['commit_total_kb'] = $totalCommitKb
        [pscustomobject]$objProps
    }
}

$tmpDir = Join-Path $PSScriptRoot 'notepad'
Remove-Item -Path $tmpDir -recurse -force -ea 0 | out-null

get-process -name 'notepad' -ea 1 | New-VmMapLogFile -DestinationPath $tmpDir
Import-VMMapLogFiles -Verbose -DirectoryPath $tmpDir

results

from gui

script output

PID                           : 15320
Process                       : "C:\Windows\System32\notepad.exe"
free_kb                       : 135287120640
free_commit_kb                : 0
private_data_kb               : 4238760
private_data_commit_kb        : 316
unusable_kb                   : 2904
unusable_commit_kb            : 0
thread_stack_kb               : 512
thread_stack_commit_kb        : 80
heap_(shareable)_kb           : 64
heap_(shareable)_commit_kb    : 64
shareable_kb                  : 2147510224
shareable_commit_kb           : 46412
mapped_file_kb                : 23040
mapped_file_commit_kb         : 23040
heap_(private_data)_kb        : 4516
heap_(private_data)_commit_kb : 636
image_(aslr)_kb               : 52720
image_(aslr)_commit_kb        : 52720
commit_total_kb               : 123268