Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a process that outputs a line for every progress update (sidenote: it does clear/replace the line, no pure newline break).

I want to save the latest line of that process to an output file or truncate the output file to keep the size manageable.

At the moment I have genrtr > genrtr.log and with a cron I tried to use > genrtr.log but it doesn't work. Also rm genrtr.log doesn't help because then the process stops updating the file.

I understand why those don't work, but wonder how to restructure it so it fits my needs.

Tried genrtr | sed -ne '$w genrtr.log' but then it waits for the process to end before writing to the file.

Clarifications: The process produces output every 1 second and unless the server crashes the process will keep running for ever.

share|improve this question
    
What exactly is genrtr? Can you edit it? –  terdon Apr 13 '13 at 15:01
    
It's a program written in C that I can edit but have almost no knowledge on C and I would rather not mess with that for maintenance purposes. –  Sev Apr 13 '13 at 16:54
    
Maybe logrotate can help you? I do not have enough experience with that, so I don't know whether that's what you are looking for. –  ignis Apr 19 '13 at 18:31
add comment

3 Answers

i was wondering whats the frequency of the run ? ....

try tee to redirect output to the logfile , its a utility often used to make a copy of the strem and redirect that copy into the output file

use : 
  command | tee command_result.log 

you could have a monitor function in a wrapper script that invokes this program which will delete the top 10 lines after some intreval ....... this way your log file wont get more than a few KB

also if there are useless spaces in between you could use translate "tr" utility to squeeze ... example :

Nitin@Kaizen ~
$ df -h | head -1
    Filesystem      Size  Used Avail Use% Mounted on

Nitin@Kaizen ~
$ df -h | head -1 | tr -s ' '
    Filesystem Size Used Avail Use% Mounted on   *** note the squeeze in space 

hope this helps

share|improve this answer
    
I didn't know of tee, but it seems to have the same effect as > while it adds data while the process is running, when I > command_result.log the file is unaffected. –  Sev Apr 13 '13 at 18:24
    
what tee does is that it duplicates the stream , but here it might help :: command | tee lofile 2>&1 > /dev/null .... this way you get a seperate copy of the log that you can manipulate in your wrapper script to extract usefiull info .... or whatever , the dev/null is a special file that dosent grow .... so your space issue is also managed this way. –  kabirz.domain Apr 18 '13 at 5:40
add comment

There are a few things you could try. The easiest is ti just print the last lines of the file:

tail genrtr.log

Then, once the process finished delete the log file. Another option, would be to periodically overwrite the file.

  • Launch the process in the background:

    genrtr > genrtr.log &
    
  • Overwrite the contents of the logfile:

    echo > genrtr.log
    

The file is now truncated but will continue to be updated by gentr so the next update report will be written to it. You could automate this, for example, truncating the file if it gets larger than 1MB:

while true; 
  do if [ $(stat -c%s genrtr.log) -gt 1000000 ]; then 
    tail genrt.log > /tmp/foo && cat /tmp/foo > genrt.log; 
     fi;
done

That little scriptlet will run until you stop it (while true;), and every time that genrtr.log is larger than one MB, it will keep the last few lines and delete the rest of the file.


UPDATE:

As Scott very correctly pointed out below, if your output contains \r to clear the line, tail will not work as expected. This, however, should:

while true; 
  do if [ $(stat -c%s genrtr.log) -gt 1000000 ]; then 
    tail genrt.log | perl -pe 's/.+\r(.+)/$1\n/' > /tmp/foo && cat /tmp/foo > genrt.log; 
     fi;
done

The perl command deletes everything before the last \r and prints the last "line" (the data after the \r and a newline. The result should be that the list line is kept, the rest of the file is cleared and the file continues to be populated.

share|improve this answer
    
Unfortunately the ` > ` redirect doesn't seem to allow any modification to the target file while the process is running and genrtr will run for ever. –  Sev Apr 13 '13 at 18:53
1  
@terdon: The question says that the program doesn’t end the update messages with newlines, but rather with escape sequences to erase and re-use the same line on the screen. Do you believe that tail will behave intelligently with that input? (Maybe with tail -400c, to get the last 400 characters and hope that that is at least the last five lines.) –  Scott Apr 15 '13 at 21:12
1  
@Sev: We seem to be having a failure to communicate. Are you saying that, if you type genrtr > genrtr.log, the output still comes out on the screen, and genrtr.log is empty? If so, what happens if you type genrtr 2> genrtr.log? –  Scott Apr 15 '13 at 21:14
    
genrtr > genrtr.log doesn't show output on the screen, it redirects the output correctly to the file, but while the process is running, you can't amend the file successfully (tried > genrtr.log and similar) I tried genrtr 2> genrtr.log and it saved the top info text of the initial output on the file, but the main repeating progress update text shows up on the screen instead. –  Sev Apr 16 '13 at 9:13
1  
Thanks @Scott, will remember from now on. –  Sev Apr 17 '13 at 22:50
show 3 more comments

I believe that this is going to be very tricky to do without either modifying genrtrc or writing a new program.  If you’re more comfortable doing the latter, I suggest this outline:

int   c;
FILE  *fp;

fp = fopen(log_file, "w");
if (fp == NULL) (Handle errors)
while ((c = getc()) != EOF)
{
    putchar(c);
    if (c ==escape sequence that ends a line)
    {
        fflush(fp);     //You should probably check for errors here, too.
        rewind(fp);
    }
    else
        putc(c, fp);
}

This acts like a combination of tee and tail -- reading standard input, and writing it to standard output and a file -- with the difference that it keeps only the last line in the file.  Then you would run

genrtr |the_above_program

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.