This behavior is controlled through a kernel setting called
vm.swappiness, which may be set on a range from 0 to 100. The default value of 60 will cause the system to swap somewhat before physical memory is completely full, which helps maintain performance under memory pressure, while not swapping so much as to degrade performance due to thrashing.
This can be disabled by setting swappiness to 0, in which case the system will not swap unless absolutely necessary. This may make sense if you usually have lots of free RAM. However, performance will fall off a cliff the moment your system's physical memory is completely filled as the system is suddenly forced to swap at an inopportune time when an application needs memory, rather than in advance, and is not recommended in your case as your server is running close to full.
It's probably better for you to set swappiness to a low (but nonzero) value like 10, which will reduce unnecessary swapping while not waiting until the last moment to free up RAM when a program needs it. The best setting depends on your environment, so experiment with different settings until you get the best results.
To change the swappiness setting, write the desired value to
echo as root. To make permanent changes, add a
vm.swappiness line to
/etc/sysctl.conf (replace 10 with your desired value as needed):
vm.swappiness = 10