Abnormally large size of /var/log/user.log and /var/log/syslog

Question

I recently started using multiprocessing for my python code and now these two files are choking up all the space on my system.

./var/log/user.log: 8.1G 
./var/log/syslog: 8.1G 

I know it might be difficult to get a fix for this without going into the specifics of the code, I just want to know if it is safe to delete these files?

Answer 1

These are log files that are intended to for debugging and accountability of users. On your personal computer, you can absolutely delete them.

However, while that solves the problem temporarily, multiprocessing should not make entries in these files.

What is written into these log files when you execute your program? Look for the strings and where they are written in your program, and take out these statements.

Answer 2

Not sure which distro you are using, but seems like you have a custom logging instance setup in /etc/syslog.conf.

Try to disable or, alternatively, if the logging info is useful, configure /etc/logrotate.conf to rotate those logs on a more regular basis in order to prevent them from filling up your filesystem.

Answer 3

I used this solution and I decreased my log rotate range from rotate 4 weekly to rotate 1 daily.

You can change the default behaviour in /etc/logrotate.conf, for example:

size 250M
rotate 2
create
#compress
include /etc/logrotate.d

Or specifically for the log file (e.g. /var/log/syslog):

/var/log/syslog
{
size 250M
rotate 2
# missingok
# notifempty
# delaycompress
# compress
# postrotate
# invoke-rc.d rsyslog reload > /dev/null
# endscript
create
}

Answer 4

When you delete these files the space won't be freed until the processes holding them open have exited. In other words, if your goal is to free up the space you'll also have to restart syslogd (which might not be easy, depending on your system)

You can see which processes are holding open the files using something like:

lsof|egrep '/var/log/(user.log|syslog)'