How can I increase the memory allocated to a process in Ubuntu server

Question

I am trying to set up a highly concurrent messaging server and it appears to use a lot of memory. Currently our server OS is Ubuntu and server software is written in Java. Under load we get out of memory exception but only 70% of my 8GB RAM is used.

How can I allocate more memory to my Java process?

Answer 1

As far as I can tell, there are four possibilities:

1. Ubuntu doesn't limit the memory per process by default, but there are ways to set up those limits. The easiest way of knowing what has been done (if anything) would be consulting the person who administrates the server.

2. If you're on a VPS and not on an actual server, there might be a problem with the configuration of the VM.

3. I don't know exactly how Java works, but C restricts the memory available to the stack when compiling. In this case, you can either instruct the compiler to increase the stack size or move larger arrays to the heap.

4. If your OS is 32-bit, the available memory per process is hard-limited (somewhere between 2 GB and 4 GB). If your server has a 64-bit CPU, that can be fixed easily by installing a 64-bit OS.

Possibility 4 is pretty easy to rule out. To verify whether it's 1, 2 or 3, you could compile the following with gcc (or write the equivalent in Java):

#include <stdlib.h>
#include <string.h>

int main(int argc, char **argv)
{
    long bytes = atol(argv[1]), counter, *pointer, words = bytes / sizeof(long);

    // try to allocate memory

    if ((pointer = malloc(bytes)) == NULL)
    {
        puts("Could not allocate memory");
        exit(1);
    }

    // play with memory, so compiler won't optimize allocation out

    for(counter = 0; counter < words; counter++)
        pointer[counter] = counter;

    exit(0);
}

and execute it the the amount of memory you want to allocate as a command-line option.

You should be able to allocate all memory that's reported as free or cached. If you can, you rule out possibility 1 and possibly 2.

Answer 2

The largest memory used by java processes is the heap, where most java objects are stored. By default, it is limited to either 1 GB or a fraction of memory present. When you are out of heap space, you got a java out of memory exception. You can increase (or decrease) the size of the heap for a JVM by using the -Xmx option. e.g.:

java -Xmx=2048m ...

If you are using a 32 bit JVM, you cannot reliably set it to more than ~2.5 to 3 GB. You need to switch to a 64 bit JVM to avoid this limitation.