I am having issues with a server running out of physical memory and I'm having troubles discerning whether it is from my application's Java process or something else on the server. Let's take the following scenario:

Server physical memory: 3747MB
Java -Xms64m
Java -Xmx512m
Java XX:MaxPermSize=512m

When I boot up the server, the OS (RHEL) reports that 487MB are being used, using your favorite memory reporting tool (top, cat /proc/meminfo | grep Mem, free -m, etc). When I start my Java process (pid 123), it uses around 215MB of physical memory (as reported by RES memory in ps -f -p 123), taking my total used memory up to around 700MB.

If I let it run for an entire day, the RES memory for my process fluctuates a little, but is generally consistent. However the total server memory has steadily increased around 1500MB, taking it to a total of 2200MB.

If my java heap size or perm gen heap were growing, wouldn't it be reflected in the process' RES memory?

Also, I can't seem to account for that extra 1500MB anywhere.

# ps aux | awk '{ RES+=$6 } END { printf("RES: %.2fMB\n", RES/1024) }'
RES: 722.23MB

Can anyone help me find that lost memory? I am basically trying to figure out if this is my problem with the application, or the infrastructure team's problem with their server build.

migrated from stackoverflow.com Jan 16 '13 at 10:57

This question came from our site for professional and enthusiast programmers.

  • 6
    Do you count disk cache? If you use free -m, you should look at the -/+ buffers/cache row. – yiding Jan 16 '13 at 4:10
  • You may use visualvm and see details of java process.. Are there any other process running and taking memory ? – Jayan Jan 16 '13 at 4:28
  • 3
    A running Linux system will quickly end up allocating all "free" memory to disk cache. This memory is still free because it can be reclaimed from cache and given to processes without any delay. – Ex Umbris Jan 16 '13 at 7:16
  • It does appear that my missing memory is taken by the disk cache. I am still puzzled, though, because my process will eventually die, without any errors or exceptions. I will need to adjust my metrics to gather more data. Thanks! – schmimd04 Jan 16 '13 at 14:18

Linux uses the policy of reclaiming, but not marking as "really free" any recently used memory (on the theory that scrubbing it costs effort, leaving the stuff around in case somebody uses it again costs nothing and may save a bundle). Don't worry about "free memory" reports. Look at how much (if any) swap is being used (swap is essentially a disk space for memory requirements that really overflow physical memory; disk is extremely slow, you don't want to need it). If you are worried about performance, install and configure monitoring software like the infamous sar (sysstat, surely there is a package for your system), it will record what is going on in minute detail for later perusal. With said reports in hand you'll know what (if anything) is your bottleneck. The quip that "Premature optimization is the root of all evil" is because people are notoriously bad at guessing where the real performance problems are, and end "fixing" something that is working perfectly fine.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.