93

I see so many guides on how to run crontab, but what I need right now is to learn how to

  1. Find log files about cron jobs
  2. Configure what gets logged
1

7 Answers 7

88

Check if the programs you run with cron have their own log files. If if they don't but write their output to the standard outputs you can redirect these to files or mail them to you. Inside crontabs standard shell redirection works.

E.g. to redirect the error output of some_job.sh to some_job.err and discarding the standard output (i.e. sending it to /dev/null) add the following redirection to your crontab

 33 3 * * * /path/to/some_job.sh 1> /dev/null 2> /other/path/to/some_job.err

or to mail it to you instead (if mail is available)

 33 3 * * * /path/to/some_job.sh 1> /dev/null 2>&1 | mail -s "cron output" you@example.org
3
  • This is how I've always solved my cron issues. Mar 21, 2010 at 4:47
  • Is that second example right? Wouldn't it redirect everything to /dev/null?
    – Timmmm
    Nov 25, 2012 at 17:44
  • 8
    @Timmmm: I does work. It first redirects everything from some_job.sh's stdout to /dev/null and only after its stderr to stdout (which now contains nothing itself anymore). That way only its stderr ends up in stdout and is passed on to mail. Nov 25, 2012 at 18:34
10

Most cron daemons on platforms I've worked with automatically email the stdout/stderr of user cron jobs to the user whose crontab the job came from. I forget what happens to system-wide (non-user-specific cron jobs from /etc/crontab). The thing is people don't always set up a mailer daemon (that is, a Mail Transfer Agent (MTA) like sendmail, qmail, or postfix) on most Unix-like OSes anymore. So the cron job output emails just die in a local mail spool folder somewhere if they even get that far. So one answer might just be to fire up your mailer daemon, and maybe make sure you have a ~/.forward file to forward your local mail along to your "real" email account.

If you want your jobs to write to specific log files, you can use standard output redirection like @honk suggested, or, supposing your cron job is a shell script, you could have your script call logger(1) or syslog(1) or whatever other command-line tool your OS provides for sending arbitrary messages to syslog. Then you could use your OS's built-in methods for configuring which kinds of messages get logged where, perhaps by editing /etc/syslog.conf.

Most of my cron jobs invoke bash scripts I wrote specifically for the purpose of being started by cron for a particular reason. In those, especially when I'm initially writing and debugging them, I like to use bash's "set -vx" to make the unexpanded and expanded form of each line of the shell script get written to stdout before it gets executed. Note that shell scripts started from cron are considered non-login, non-interactive shells, so your standard shell startup scripts like .bashrc and .profile aren't run. If you use bash and want bash to run a startup script, you have to define an environment variable "BASH_ENV=/path/to/my/startup/script" in your crontab before the line where you define the job.

3
  • 3
    This is correct. One can use mail command to read messages from the command line. Or look at /var/spool/mail. But if you have installed postfix or other mailer than standard sendmail, another way to read messages is needed. Feb 19, 2015 at 16:44
  • 1
    Why do we need a mail service agent just for this? mail -s "cron output" test@example.com works just fine :/ Mar 21, 2017 at 19:54
  • On CentOS, my cron output gets "mailed" to /var/spool/mail. See it by running less $MAIL if you want to see cron output for the current user or less /var/spool/mail/root if you want to see cron output for commands running as root.
    – sffc
    Jul 7, 2017 at 10:09
5

The tasks cron is executing are responsible for their own logging.

0
5

I think redirecting within the cron-file might not be the best option in this case.

Often you want the logging speification co-located with the cron job script. In this case, I suggest the following:

#!/bin/bash
exec &>> capture-log.txt
echo "Running cron-job foo at $(date)"
...
<rest of script>

This appends the output from the cron job to the capture-log.txt file.

2
  • 1
    The &>> redirection operator is not available in sh scripts, or really old versions of Bash. The standard and portable way to express the same thing is >>capture-log.txt 2>&1 i.e. redirect standard output to append to the file and redirect standard error to the same place.
    – tripleee
    Mar 23, 2022 at 6:31
  • See also Explain the bash command exec > >(tee $LOG_FILE) 2>&1 to try to make sense of this unusual but powerful usage of exec. Aug 6, 2022 at 7:05
4

The simplest way is to capture error messages and save them in a file. I have a cron job that calls a php command line, like this:

1 0 * * * php /pathOfMyApp/index.php controllerName functionName > /pathOfMyApp/log/myErrorLog 2>&1

The part before > is my cron job and after > is the capture and save in a file located in a log folder in the root of my project, but could be any place you want. Note: every time the cron job runs it will overwrite the previous log file. You can use >> to append to the end of an existing file.

If your crontab uses curl or wget and refers to a link, you can search in /var/log/httpd/appName for access-log, if cron is returning with 500 or 400 must be something wrong.

As a last resort, you can check /var/log/messages too.

2
  • 1
    You are overwriting the output file on every invocation, which can be quite troublesome. Probably better to append with >>
    – tripleee
    Mar 23, 2022 at 6:34
  • The paragraph about /var/log/httpd/appName seems confused. I would expect that to contain errors from the HTTP server though of course if you fetch a URL from localhost the server is obviously on the same machine as the cron job.
    – tripleee
    Mar 23, 2022 at 6:38
3

I prefer to get e-mail reports about cron jobs. Just put

MAILTO=yourmail@domain.com

into crontab and you will get an email. Of course you have to have e-mail configured for your account.

Additional information: You can put MAILTO before each line in your crontab to specify different e-mails:

MAILTO=mail1@domain.com
0 1 * * *     <job at 1 a.m. sent to mail1>
MAILTO=mail2@domain.com
0 2 * * *     <job at 2 a.m. sent to mail2>
5
  • I find this to often be the easiest to turn on and off for debugging. I typically just use username@localhost for the email address.
    – flickerfly
    Jul 18, 2014 at 20:15
  • where to put that? Sep 15, 2020 at 7:32
  • Put is at the top of your crontab. When you enter crontab with crontab -e. Put MAILTO=<e-mail> line as the first line, and it will apply for all crontab entries. You can also specify MAILTO for each entry, just put different MAILTO=<email> in front of each crontab entry/line.
    – nobody
    Sep 16, 2020 at 8:50
  • This obviously requires mail to be properly configured on the system, which is often not the case, especially in ad-hoc Docker containers etc. If you don't have a MTA installed at all, maybe look for a dead.letter file in your home directory; though modern cron implementations will detect this, and simply discard the output if it can't be sent anywhere.
    – tripleee
    Mar 23, 2022 at 6:33
  • @tripleee It is written in the answer that you have to have your e-mail configured. Thanks for the dead.letter and other info.
    – nobody
    Mar 23, 2022 at 8:43
0

How to easily log all output from any executable

...what I need right now is to learn how to

  1. Find log files about cron jobs
  2. Configure what gets logged

Summary

Put this magic line at the top of any script, including inside your scripts or wrapper scripts which get called as cron jobs, in order to have all stdout and stderr output thereafter get automatically logged!

exec > >(tee -a "$HOME/cronjob_logs/my_log.log") 2>&1

Details

Cron jobs are just scripts that get called at a fixed time or interval by a scheduler. They log only what their executables tell them to log.

But, what you can do is edit the call command as listed in your crontab -e file in such a way that it causes all output to get logged when you call it.

There are a variety of approaches, such as directing it to a file at call time, like this:

# `crontab -e` entry which gets called every day at 2am
0 2 * * * some_cmd > $HOME/cronjob_logs/some_cmd.log 2>&1

But, what if I want it to automatically log when I call it normally to test it, too? I'd like it to log every time I call some_cmd, and I'd still like it to show all output to the screen as well so that when I call the script manually to test it I still see all output! The best way I have come up with is this. It looks really weird, but it's very powerful! Simply add these lines to the top of any script and all stdout or stderr output thereafter will get automatically logged to the specified file, named after the script you are running itself! So, if your script is called some_cmd, then all output will get logged to $HOME/cronjob_logs/some_cmd.log:

# See my ans: https://stackoverflow.com/a/60157372/4561887
FULL_PATH_TO_SCRIPT="$(realpath "${BASH_SOURCE[-1]}")"
SCRIPT_DIRECTORY="$(dirname "$FULL_PATH_TO_SCRIPT")"
SCRIPT_FILENAME="$(basename "$FULL_PATH_TO_SCRIPT")"
# See:
# 1. https://stackoverflow.com/a/49514467/4561887 and 
# 2. https://superuser.com/a/569315/425838
mkdir -p "$HOME/cronjob_logs"
exec > >(tee -a "$HOME/cronjob_logs/${SCRIPT_FILENAME}.log") 2>&1 # <=MAGIC CMD

Here are the relevant references from above to understand it all:

  1. [my answer] How to obtain the full file path, full directory, and base filename of any script being run OR sourced...
  2. Explain the bash command exec > >(tee $LOG_FILE) 2>&1 - "process substitution"
  3. How can I view results of my cron jobs? - shows something similar to the "process substitution" above

Let's see this in full context, with a little beautification and a bunch of explanatory comments, where I am wrapping some_executable (any executable: bash, C, C++, Python, whatever) with a some_executable.sh bash wrapper to enable logging:

some_executable.sh:

# -------------- cron job automatic logger code START --------------

# See my ans: https://stackoverflow.com/a/60157372/4561887
FULL_PATH_TO_SCRIPT="$(realpath "${BASH_SOURCE[-1]}")"
SCRIPT_DIRECTORY="$(dirname "$FULL_PATH_TO_SCRIPT")"
SCRIPT_FILENAME="$(basename "$FULL_PATH_TO_SCRIPT")"

# Automatically log the output of this script to a file!
begin_logging() {
    mkdir -p ~/cronjob_logs

    # Redirect all future prints in this script from this call-point forward to
    # both the screen and a log file!
    #
    # This is about as magic as it gets! This line uses `exec` + bash "process
    # substitution" to redirect all future print statements in this script
    # after this line from `stdout` to the `tee` command used below, instead.
    # This way, they get printed to the screen *and* to the specified log file
    # here! The `2>&1` part redirects `stderr` to `stdout` as well, so that
    # `stderr` output gets logged into the file too.
    # See:
    # 1. *****+++ https://stackoverflow.com/a/49514467/4561887 -
    #    shows `exec > >(tee $LOG_FILE) 2>&1`
    # 1. https://superuser.com/a/569315/425838 - shows `exec &>>` (similar)
    exec > >(tee -a "$HOME/cronjob_logs/${SCRIPT_FILENAME}.log") 2>&1

    echo ""
    echo "====================================================================="
    echo "Running cronjob \"$FULL_PATH_TO_SCRIPT\""
    echo "on $(date)."
    echo "Cmd:  $0 $@"
    echo "====================================================================="
}

# --------------- cron job automatic logger code END ---------------
# THE REST OF THE SCRIPT GOES BELOW THIS POINT.
# ------------------------------------------------------------------

main() {
    some_executable
    echo "= DONE."
    echo ""
}

# ------------------------------------------------------------------------------
# main program entry point
# ------------------------------------------------------------------------------
begin_logging "$@"
time main "$@"

There you have it! Now isn't that pretty and nice!? Just put whatever you want in the main() function, and all of the output there gets automatically logged into $HOME/cronjob_logs/some_executable.sh.log, whether you call this wrapper manually or whether it is called by a cron job! And, since the tee command is used, all output goes to the terminal too when you run it manually so you can see it live as well.

Each log entry gets prefixed with a nice header too.

Here is an example log entry:

=========================================================================================
Running cronjob "/home/gabriel/GS/dev/eRCaGuy_dotfiles/useful_scripts/cronjobs/repo_git_pull_latest.sh"
on Wed Aug 10 02:00:01 MST 2022.
Cmd:  REMOTE_NAME="origin" MAIN_BRANCH_NAME="main" PATH_TO_REPO="/home/gabriel/GS/dev/random_repo" "/home/gabriel/GS/dev/eRCaGuy_dotfiles/useful_scripts/cronjobs/repo_git_pull_latest.sh" 
=========================================================================================
= Checking to see if the git server is up and running (will time out after 40s)...
= git server is NOT available; exiting now

real    0m40.020s
user    0m0.012s
sys 0m0.003s

You can find and borrow this whole script from my eRCaGuy_dotfiles repo here: repo_git_pull_latest.sh.

See also my cronjob readme and notes here: https://github.com/ElectricRCAircraftGuy/eRCaGuy_dotfiles/tree/master/useful_scripts/cronjobs

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.