Swap partition size on a 64 GB RAM computer for memory-intensive work

Question

I have 64 GB RAM and 240 GB SSD on my computer, which I'm going to use for memory-intensive calculations (machine learning, data mining, etc.). Most of the advice I found on the Internet are about 2-4-8 GB RAM computers, and they recommend 2x the size of the RAM swap partition (so 128 GB).

Is it reasonable to make a 128 GB swap partition? what advantages do I get by making a huge swap partition?

Do I understand correctly that, in case I run out of physical RAM:

1. If I have no swap, I get an 'out of memory' error
2. If I do have swap, some of RAM pages will be copied to swap partition, and the program will continue to run (although more slowly).

Some people say it's a bad idea to make swap on SSD, since it has limited amount of read/write cycles. How fast using swap will it exhaust SSD read/write cycles (as far as I know, it's about 50000 write IOPS)?

I'm using Linux (Ubuntu 14.04 (Trusty Tahr)).

Going to set a 16 GB swap for now, as it should be surely enough (for example, RedHat suggests 4 GB), and 16 GB of disk space isn't actually a big deal.

Answer 1

You probably only need a small amount of swap. When you have sufficient RAM for your computer's typical working set, which I'm pretty sure you do, you only need swap for two things:

1. You need swap to get information that will likely never be accessed out of RAM to free up more space for disk cache. Many applications run on system startup and will never be accessed again. You don't want any pages they dirtied stuck in RAM forever. So you need swap to hold them.

2. You need swap to cover allocations that will never be filled. This space simply has to be available, even though it will not be used. Without it, the system will have to refuse to allocate memory even when it has plenty of free physical RAM because it has insufficient backing store to permit all of its allocations to be used at once.

Neither of these requires a large amount of swap. 16GB, for example, should be more than enough. The purpose is not to let you run bigger working sets at the cost of speed. The purpose is to let you use your 64GB effectively and not have to clog it with junk or reserve it for edge cases that will never happen.

(I agree with Bert that 4GB is quite likely to be sufficient.)

Answer 2

RedHat recommends 4 GB on a machine with 64 GB.

However, sizing swap is more of an art than a science. It depends on what the machine is being used for, how much disk space and memory you have, and other factors. Remember, you can always add more swap later.

Using the 2X physical memory rule is outdated with the amount of memory systems have these days. But running with zero swap is not recommended unless you know what you are doing. The recommendation of 4 GB is a good starting point.

Answer 3

On Linux, you need enough swap so that the total virtual memory available (RAM + SWAP) is enough for all the processes you want to run at once and their maximum virtual footprint.

If you have less swap than this, or no swap at all, then the following situation happens: the system runs out of memory trying to allocate a page. But, this is still a soft failure even though there is no swap, because the system has plenty of "victim" pages that can be removed in order to make room: namely, the pages of all file-backed memory mappings, such as executables and shared libraries!

As your system demands more and more space for data (which cannot be swapped out), it will increasingly evacuate the executable code (shared libraries and executables), leading to terrible thrashing, as the working set is trimmed into a tighter and tighter set of pages.

Swap space softens this problem by providing a place for anonymous (not file mapped) pages to be swapped out: the pages used for memory allocations, so that executable code can stay in memory.

Even so, if you don't frequently run memory-intensive tasks, you may be able to get away with running swapless most of the time, and manually configure a swap file (instead of a dedicated partition) when you need it. To make a swap file on the fly, become root and:

dd if=/dev/zero of=/path/to/swapfile size=$((1024 * 1024)) count=32768  # 32 Gb.
mkswap /path/to/swapfile
swapon /path/to/swapfile

When you don't need it any more:

swapoff /path/to/swapfile
rm /path/to/swapfile

Notes:

1. You definitely do not need to configure at least as much swap as you have RAM. This rule of thumb dates back to operating systems where it was a hard requirement due to the way swapping was designed.

2. There are ways to make Linux fail hard when no memory is available, namely via manipulating the values of these sysctl entries:

   vm.overcommit_memory
   vm.overcommit_ratio
   

Answer 4

There are more considerations. If you need/want suspend to work then you need at least the size of your RAM and then some. However it sounds unlikely you need it given that you seem to mainly build a computational work horse.

In this case please consider using a swap file instead of a partition. You don't need to worry about sizing much, getting rid of or adding it later doesn't require any repartitioning. There is no (noticable) performance penalty using a file over a partition. If you ever happen to need it, have a look at the size and this will then also give you good hints.

Answer 5

The workload you want to apply to the machine needs a certain amount of memory to run (remember to add enough to the equation to handle peak loads), and you need to configure your computer to have at least that.

Modern operating systems provide virtual memory as a combination of physical memory and swap space, so if you need more memory than the machine has available you must add enough swap space to fill the gap. I.e. if you need 80 GB max, and the machine has 64 GB you need 16 GB swap.

Typically operating system installers ask you to make an initial swap area (as this is the simplest and allow even tiny computers to install), and it has been found that a good rule of thumb for typical Unix operations is to have virtual memory sized to be three times the physical memory, so this is typically suggested. You, however, know more about the usage pattern so you can change this as appropriate.

There is nothing wrong with working without swapspace if your memory pressure always is less. Linux will transparently use any unused memory as a disk cache.

Answer 6

A much better idea than having "a lot of swap" is (re-)organizing your work so that the working sets fit in memory, then using the file-system to store and retrieve the work you do. I.e., instead of forcing the OS to guess what your memory usage patterns will be, use what you know about your problems to control your memory usage patterns.

As a random example that is immediately relevant to me this summer... In implementing the quadratic sieve, one needs a large (apparently) contiguous array to mark up (with some complicated algorithm whose details actually don't matter for this example). The array needs to be ~100 Giga-entries, so easily in the 1 TB range. I could pretend to allocate that and let the OS do an amazing amount of inefficient swapping to get pages in and out of RAM to support all the sequential writes through the array. Instead of doing something that boneheaded, I have arranged to allocate a much smaller array that exactly fits in memory and then reuse that little array to iteratively cover the rest of the big array in slices. I've also stripped the OS, stripped the running set of services, replaced the shell, and customized two layers of memory allocators to do their darnedest to keep as much of the address space available to my process as close to contiguous as possible.

SSD may be fast, but it is not nearly as fast as organizing your computation to do the same set of operations without ever stalling to swap.

Answer 7

As the others mentioned, a swap partition is a good idea even if you have plenty of RAM. It's not a good idea to put it on an SSD; the frequent writes of a swap partition will eventually wear out your drive.

If you have a spare USB 3.0 port, I would suggest using a flash drive for your swap space. There are plenty of high-speed flash drives that are as fast as your SSD, but much cheaper - cheap enough to replace if it did begin to fail. A quick search on Amazon shows that there are many decent 16 GB USB 3.0 flash drives for under $20, and even some 64 GB drives under $60.

Partition the entire flash drive as swap space, and you'll have swap space if you need it, and peace of mind knowing that the memory being repeatedly written is easily (and cheaply) replaceable.

Answer 8

You will be fine even with 1GiB (and likely less) of swap. My work computer typically uses no more than 140-150 MiB. A gigabyte is plenty of over-provisioning for that.
Unless you do compute tasks that require datasets in the hundreds of gigabytes and (this one is important!) data is accessed in a more or less access-once fashion, you will never want to have a swap much larger than that. But then again, simply memory mapping a datafile works equally well for that application.

But more swap helps more, right? More of anything is always better!

Consider what difference a swap of, say, 16GiB will make (or think of 64GiB). If you never use these 16GiB, you could as well not have them set aside in the first place. But if you do use them, what happens? Disk, compared to main memory, is exceedingly slow. Even with a SATA-600 SSD, transferring 16GiB takes between 30 and 40 seconds, and 2-4 times as long on some other configurations.
Now someone will inevitably object that you are rather paging in and out a dozen or so 4kiB pages, not 16GiB in one go. While that is true, the point nevertheless stands. If you only need to swap in and out a couple of pages, you don't need 16GiB of swap, but if you do need 16GiB of swap, then you are going to transfer them, too (one way or another).

In theory, 99.9% of all users could even use a 64GiB machine (or any 8+GiB machine) without any swap, and most likely never notice something missing. However, this is not adviseable.
First, it is sub-optimal because the operating system has fewer choices in what it can discard when it runs out of physical memory. There are two things it can do: Swap out something that isn't used, or throw away pages from the buffer cache. If you have no swap, there is only one thing it can do. Throwing away pages from the buffer cache is harmless, but it may noticeably impact performance.
Second, private anonymous mappings might simply fail if there is no swap. That usually won't happen, but eventually when there is not enough physical memory available to satisfy them all, and there is no swap, the operating system has only either this choice, except...
Third, the dreaded OOM killer may kick in. Which means a more or less random process gets to get killed. No thank you. This is not something you want to have happening.

With that said, advice such as you need a swap X times the amount of RAM installed comes from people who repeat something they heard (and didn't understand!) from someone who repeated something they heard (and didn't understand!) decades ago.
The "use 2X your RAM" rule was an easy to remember rule of thumb in the 1980s and 1990s, it was never the "golden truth" (just something that worked OK for most users), and it doesn't apply at all nowadays.

You should have a reasonable amount of swap which you can easily afford (say, a gigabyte), so the OS can page out some stale stuff, and so the world doesn't immediately end when you once ask for a little more memory. But that's it.