Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a Linux program which can write information to stdout and stderr.

I have a shell script which redirects that output to a file in /var/log. (Via >> and 2>&1.)

Is there a way to make that log file rotate? (max size, then switch to a different file, keep only a limited number of files)

I've seen a few answers which talk about the logrotate program, which sounds good, but they also seem to be focused on programs which are generating log files internally and handle HUP signals. Is there a way to make this work with a basic output redirection script?

share|improve this question
    
Why can't you just modify the script that redirects the output to contain the logic for the rotation? –  MaQleod Jun 1 '11 at 5:05
    
I could, if someone could tell me how to detect the size of a logfile and rotate it out from under the stdout of a process without disturbing that process. I don't have to use logrotate if there's a better option, that just sounded like a convenient starting point for discussion. –  Miral Jun 1 '11 at 5:12
1  
You don't have to use logrotate, but using logrotate just saves time... There is usually little point reinventing the wheel. –  bubu Jun 1 '11 at 5:43
    
Exactly my point. So is there a way to make logrotate work with an ongoing process's redirected stdout? –  Miral Jun 1 '11 at 6:12

6 Answers 6

up vote 11 down vote accepted

As an alternative, you could pipe the output through tools designed with the primary purpose of maintaining size-capped, automatically rotated, log file sets, such as:

Tools to then process multilog-format log file sets include, amongst others:

share|improve this answer
    
Thanks, multilog looks like just what I needed. –  Miral Jun 3 '11 at 1:36

If you can have it go to one of the standard log streams (syslog, daemon, cron, user, security, mail, etc.) you can use the logger command and pipe to it instead.

echo "Hello." | logger -p daemon.info

Otherwise, you may be better off piping your logged content to a custom program or script to handle it, or look at setting up the logrotate configuration.

EDIT: JdeBP's answer seems to have what you may be looking for.

share|improve this answer
1  
+1 for simplicity. BTW, you can also configure a custom facility (local0) instead of the standard ones (daemon in your example) –  Roger Keays Oct 9 '12 at 7:56

the rotatelogs tool shipped with apache (in the bin dir) (see docs) takes input from stdin and rotates the log after some specific amount of time

share|improve this answer

So is there a way to make logrotate work with an ongoing process's redirected stdout?

Yes! Check out the "copytruncate" directive offered by logrotate. Specifying that instructs logrotate to handle this very situation: a simple program that keeps its log file open indefinitely.

One caveat may or may not be a problem in your situation:

Note that there is a very small time slice between copying the file and truncating it, so some logging data might be lost.

Anecdotally, I've seen some "real world" log sources that do encourage users to apply this directive. There's some discussion of this option here.

share|improve this answer

I had similar problem and had initially discard logrotate but it turned out logrotate can actually do this well, the key directive is "copytruncate". For some reason that term didn't come up on any of the googling I did, so I am adding this answer to clarify exactly how to use it for this case.

The trick is this only works if the redirect is done with ">>" (append) instead of ">" (create).

Config File (truncate.cfg):

/tmp/temp.log {
    size 10M
    copytruncate
    rotate 4
    maxage 100
}

Test Program (never gives up file). You can watch it filling disk and though deleting logfile will appear to work it will not actually free up any space on the disk:

echo /dev/urandom >> /tmp/temp.log

Running log rotate:

logrotate truncate.cfg
share|improve this answer
    
It's a nice theory, but it doesn't actually work on any system I've tried it on. The file does not actually get truncated and the program continues to append to it as before. (And yes, that's even with the redirection done via >>.) ((BTW, this answer was already given previously.)) –  Miral yesterday
    
… as discussed in logrotate won’t truncate original file (on our Unix&Linux site). Also, echo /dev/urandom >> /tmp/temp.log will write 13 deterministic characters to /tmp/temp.log and then immediately exit. Did you mean cat /dev/urandom? –  G-Man yesterday

I made simple log.py script. Which write to 1 file and rotate it every day and every 5M

https://github.com/axet/home-bin/blob/common/log

[note to moderators: do not inline my work without copyright notices]

# echo test 2>&1 | log ./run.log
#
-rw-r--r-- 1 axet axet     850176 Feb 22 12:28 run.log
-rw-r--r-- 1 axet axet    1048595 Feb 22 09:00 run.log.1
-rw-r--r-- 1 axet axet    1319926 Feb 22 04:57 run.log.2
-rw-r--r-- 1 axet axet    2400918 Feb 22 01:17 run.log.3
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.