Linux memory usage doesn't add up

Question

I try to monitor the memory usage in my ubuntu server. When I run 'top' command, it shows 70% of the memory is used. (shown in the 'Mem:' line). But I want to find the processes that take my memory usage and add them up, it's far from 70%. Can anyone help explain that for me?

Thanks a lot!

Here is my top command result:

top - 20:59:06 up 23 days, 33 min,  3 users,  load average: 0.06, 0.12, 0.09
Tasks:  92 total,   1 running,  91 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.3%us,  0.2%sy,  0.0%ni, 99.5%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   4124236k total,  2903104k used,  1221132k free,   147508k buffers
Swap:  4192252k total,      440k used,  4191812k free,  2596860k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
30558 mysql     20   0  311m  36m 5956 S    0  0.9   0:00.74 mysqld
28848 jiechao   20   0 50688 9244 3144 S    0  0.2   0:05.65 python
  956 root      20   0 11712 8004 2964 S    0  0.2   0:39.26 ddclient - slee
28846 jiechao   20   0 49256 7652 3280 S    0  0.2   0:07.17 python
22284 jiechao   20   0 10400 6876 1664 S    0  0.2   0:00.49 bash
30433 jiechao   20   0 10344 6776 1620 S    0  0.2   0:00.46 bash
31053 jiechao   20   0 10344 6768 1612 S    0  0.2   0:00.47 bash
28847 jiechao   20   0 48168 6096 2980 S    0  0.1   0:00.59 python
29178 www-data  20   0  228m 5392 2052 S    0  0.1   0:00.06 apache2
29179 www-data  20   0 12796 5148 1920 S    0  0.1   0:00.02 apache2
29180 www-data  20   0 12796 5148 1920 S    0  0.1   0:00.02 apache2
29181 www-data  20   0 12796 5148 1920 S    0  0.1   0:00.02 apache2
29182 www-data  20   0 12796 5148 1920 S    0  0.1   0:00.02 apache2
29183 www-data  20   0 12796 5148 1920 S    0  0.1   0:00.02 apache2
  999 root      20   0 11396 3948 2052 S    0  0.1   0:21.21 apache2
  926 whoopsie  20   0 24572 3284 2280 S    0  0.1   0:00.18 whoopsie
30933 root      20   0  9604 3104 2484 S    0  0.1   0:00.00 sshd
22164 root      20   0  9604 3096 2484 S    0  0.1   0:00.01 sshd
30313 root      20   0  9604 3096 2480 S    0  0.1   0:00.00 sshd
24302 root      20   0  9632 3024 2428 S    0  0.1   0:00.07 sshd
  787 root      20   0  6664 2424 1988 S    0  0.1   0:00.05 sshd
    1 root      20   0  3504 1876 1276 S    0  0.0   0:01.57 init
24421 jiechao   20   0  9632 1596  992 S    0  0.0   0:00.00 sshd
  812 syslog    20   0 31044 1484 1008 S    0  0.0   1:47.00 rsyslogd

Answer 1

Sure that number is a combination of RAM being used by process and buffers and cache for files. Linux has a philosophy of maxing out the use of RAM for files accessed rather than having to hit the hard drive for these files.

Try either using the command free or looking at the kernel itself via this command, cat /proc/meminfo.

free

% free
             total       used       free     shared    buffers     cached
Mem:       7987492    7717152     270340          0     314644    2435048
-/+ buffers/cache:    4967460    3020032
Swap:      5963772       1304    5962468

cat /proc/meminfo

% cat /proc/meminfo 
MemTotal:        7987492 kB
MemFree:          284328 kB
Buffers:          314384 kB
Cached:          2427852 kB
SwapCached:          116 kB
Active:          4698232 kB
Inactive:        2374796 kB
Active(anon):    3756264 kB
Inactive(anon):   768392 kB
Active(file):     941968 kB
Inactive(file):  1606404 kB
Unevictable:          68 kB
Mlocked:              68 kB
SwapTotal:       5963772 kB
SwapFree:        5962468 kB
Dirty:               452 kB
Writeback:             0 kB
AnonPages:       4330964 kB
Mapped:           185540 kB
Shmem:            193864 kB
Slab:             306532 kB
SReclaimable:     261520 kB
SUnreclaim:        45012 kB
KernelStack:        6192 kB
PageTables:        94376 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     9957516 kB
Committed_AS:    9671884 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      585392 kB
VmallocChunk:   34359040964 kB
HardwareCorrupted:     0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       12288 kB
DirectMap2M:     8237056 kB

The free command will show you total RAM, 7987492, in my case, of which 7717152 is being used and 270340 is free. This is a combination of the RAM, buffers and cache. The 2nd line shows you how much of that RAM is being used by the buffers and cache, 4967460, which means I really have 3020032 of RAM free.

From an itworld.com article on making sense of memory usage:

The buffers number represents in-memory blocks that result from the kernel accessing the disk, such as when the kernel needs to read the contents of files. The cached figure tells us how much RAM is being used to cache the content of recently read files. The buffer figure increases when the file system layer is bypassed while the cache grows when the file system is used. Both grow as read operations increase.

NOTE: Keep in mind that the Linux kernel's memory manager will free any RAM being used for buffers & cache is an application actually needs it.

Resources

• http://www.cyberciti.biz/faq/linux-check-memory-usage/
• Making sense of memory usage on Linux