How can I view results of my cron jobs?

Question

I see so many guides on how to run crontab, but what I need right now is to learn how to

1. Find log files about cron jobs
2. Configure what gets logged

Answer 1

Check if the programs you run with cron have their own log files. If if they don't but write their output to the standard outputs you can redirect these to files or mail them to you. Inside crontabs standard shell redirection works.

E.g. to redirect the error output of some_job.sh to some_job.err and discarding the standard output (i.e. sending it to /dev/null) add the following redirection to your crontab

 33 3 * * * /path/to/some_job.sh 1> /dev/null 2> /other/path/to/some_job.err

or to mail it to you instead (if mail is available)

 33 3 * * * /path/to/some_job.sh 1> /dev/null 2>&1 | mail -s "cron output" you@example.org

Answer 2

Most cron daemons on platforms I've worked with automatically email the stdout/stderr of user cron jobs to the user whose crontab the job came from. I forget what happens to system-wide (non-user-specific cron jobs from /etc/crontab). The thing is people don't always set up a mailer daemon (that is, a Mail Transfer Agent (MTA) like sendmail, qmail, or postfix) on most Unix-like OSes anymore. So the cron job output emails just die in a local mail spool folder somewhere if they even get that far. So one answer might just be to fire up your mailer daemon, and maybe make sure you have a ~/.forward file to forward your local mail along to your "real" email account.

If you want your jobs to write to specific log files, you can use standard output redirection like @honk suggested, or, supposing your cron job is a shell script, you could have your script call logger(1) or syslog(1) or whatever other command-line tool your OS provides for sending arbitrary messages to syslog. Then you could use your OS's built-in methods for configuring which kinds of messages get logged where, perhaps by editing /etc/syslog.conf.

Most of my cron jobs invoke bash scripts I wrote specifically for the purpose of being started by cron for a particular reason. In those, especially when I'm initially writing and debugging them, I like to use bash's "set -vx" to make the unexpanded and expanded form of each line of the shell script get written to stdout before it gets executed. Note that shell scripts started from cron are considered non-login, non-interactive shells, so your standard shell startup scripts like .bashrc and .profile aren't run. If you use bash and want bash to run a startup script, you have to define an environment variable "BASH_ENV=/path/to/my/startup/script" in your crontab before the line where you define the job.

Answer 3

The tasks cron is executing are responsible for their own logging.

Answer 4

I think redirecting within the cron-file might not be the best option in this case.

Often you want the logging speification co-located with the cron job script. In this case, I suggest the following:

#!/bin/bash
exec &>> capture-log.txt
echo "Running cron-job foo at $(date)"
...
<rest of script>

This appends the output from the cron job to the capture-log.txt file.

Answer 5

The simplest way is to capture error messages and save them in a file. I have a cron job that calls a php command line, like this:

1 0 * * * php /pathOfMyApp/index.php controllerName functionName > /pathOfMyApp/log/myErrorLog 2>&1

The part before > is my cron job and after > is the capture and save in a file located in a log folder in the root of my project, but could be any place you want. Note: every time the cron job runs it will overwrite the previous log file. You can use >> to append to the end of an existing file.

If your crontab uses curl or wget and refers to a link, you can search in /var/log/httpd/appName for access-log, if cron is returning with 500 or 400 must be something wrong.

As a last resort, you can check /var/log/messages too.

Answer 6

I prefer to get e-mail reports about cron jobs. Just put

MAILTO=yourmail@domain.com

into crontab and you will get an email. Of course you have to have e-mail configured for your account.

Additional information: You can put MAILTO before each line in your crontab to specify different e-mails:

MAILTO=mail1@domain.com
0 1 * * *     <job at 1 a.m. sent to mail1>
MAILTO=mail2@domain.com
0 2 * * *     <job at 2 a.m. sent to mail2>

Answer 7

How to easily log all output from any executable

...what I need right now is to learn how to

1. Find log files about cron jobs
2. Configure what gets logged

Summary

Put this magic line at the top of any script, including inside your scripts or wrapper scripts which get called as cron jobs, in order to have all stdout and stderr output thereafter get automatically logged!

exec > >(tee -a "$HOME/cronjob_logs/my_log.log") 2>&1

Details

Cron jobs are just scripts that get called at a fixed time or interval by a scheduler. They log only what their executables tell them to log.

But, what you can do is edit the call command as listed in your crontab -e file in such a way that it causes all output to get logged when you call it.

There are a variety of approaches, such as directing it to a file at call time, like this:

# `crontab -e` entry which gets called every day at 2am
0 2 * * * some_cmd > $HOME/cronjob_logs/some_cmd.log 2>&1

But, what if I want it to automatically log when I call it normally to test it, too? I'd like it to log every time I call some_cmd, and I'd still like it to show all output to the screen as well so that when I call the script manually to test it I still see all output! The best way I have come up with is this. It looks really weird, but it's very powerful! Simply add these lines to the top of any script and all stdout or stderr output thereafter will get automatically logged to the specified file, named after the script you are running itself! So, if your script is called some_cmd, then all output will get logged to $HOME/cronjob_logs/some_cmd.log:

# See my ans: https://stackoverflow.com/a/60157372/4561887
FULL_PATH_TO_SCRIPT="$(realpath "${BASH_SOURCE[-1]}")"
SCRIPT_DIRECTORY="$(dirname "$FULL_PATH_TO_SCRIPT")"
SCRIPT_FILENAME="$(basename "$FULL_PATH_TO_SCRIPT")"
# See:
# 1. https://stackoverflow.com/a/49514467/4561887 and 
# 2. https://superuser.com/a/569315/425838
mkdir -p "$HOME/cronjob_logs"
exec > >(tee -a "$HOME/cronjob_logs/${SCRIPT_FILENAME}.log") 2>&1 # <=MAGIC CMD

Here are the relevant references from above to understand it all:

1. [my answer] How to obtain the full file path, full directory, and base filename of any script being run OR sourced...
2. Explain the bash command exec > >(tee $LOG_FILE) 2>&1 - "process substitution"
3. How can I view results of my cron jobs? - shows something similar to the "process substitution" above

Let's see this in full context, with a little beautification and a bunch of explanatory comments, where I am wrapping some_executable (any executable: bash, C, C++, Python, whatever) with a some_executable.sh bash wrapper to enable logging:

some_executable.sh:

# -------------- cron job automatic logger code START --------------

# See my ans: https://stackoverflow.com/a/60157372/4561887
FULL_PATH_TO_SCRIPT="$(realpath "${BASH_SOURCE[-1]}")"
SCRIPT_DIRECTORY="$(dirname "$FULL_PATH_TO_SCRIPT")"
SCRIPT_FILENAME="$(basename "$FULL_PATH_TO_SCRIPT")"

# Automatically log the output of this script to a file!
begin_logging() {
    mkdir -p ~/cronjob_logs

    # Redirect all future prints in this script from this call-point forward to
    # both the screen and a log file!
    #
    # This is about as magic as it gets! This line uses `exec` + bash "process
    # substitution" to redirect all future print statements in this script
    # after this line from `stdout` to the `tee` command used below, instead.
    # This way, they get printed to the screen *and* to the specified log file
    # here! The `2>&1` part redirects `stderr` to `stdout` as well, so that
    # `stderr` output gets logged into the file too.
    # See:
    # 1. *****+++ https://stackoverflow.com/a/49514467/4561887 -
    #    shows `exec > >(tee $LOG_FILE) 2>&1`
    # 1. https://superuser.com/a/569315/425838 - shows `exec &>>` (similar)
    exec > >(tee -a "$HOME/cronjob_logs/${SCRIPT_FILENAME}.log") 2>&1

    echo ""
    echo "====================================================================="
    echo "Running cronjob \"$FULL_PATH_TO_SCRIPT\""
    echo "on $(date)."
    echo "Cmd:  $0 $@"
    echo "====================================================================="
}

# --------------- cron job automatic logger code END ---------------
# THE REST OF THE SCRIPT GOES BELOW THIS POINT.
# ------------------------------------------------------------------

main() {
    some_executable
    echo "= DONE."
    echo ""
}

# ------------------------------------------------------------------------------
# main program entry point
# ------------------------------------------------------------------------------
begin_logging "$@"
time main "$@"

There you have it! Now isn't that pretty and nice!? Just put whatever you want in the main() function, and all of the output there gets automatically logged into $HOME/cronjob_logs/some_executable.sh.log, whether you call this wrapper manually or whether it is called by a cron job! And, since the tee command is used, all output goes to the terminal too when you run it manually so you can see it live as well.

Each log entry gets prefixed with a nice header too.

Here is an example log entry:

=========================================================================================
Running cronjob "/home/gabriel/GS/dev/eRCaGuy_dotfiles/useful_scripts/cronjobs/repo_git_pull_latest.sh"
on Wed Aug 10 02:00:01 MST 2022.
Cmd:  REMOTE_NAME="origin" MAIN_BRANCH_NAME="main" PATH_TO_REPO="/home/gabriel/GS/dev/random_repo" "/home/gabriel/GS/dev/eRCaGuy_dotfiles/useful_scripts/cronjobs/repo_git_pull_latest.sh" 
=========================================================================================
= Checking to see if the git server is up and running (will time out after 40s)...
= git server is NOT available; exiting now

real    0m40.020s
user    0m0.012s
sys 0m0.003s

You can find and borrow this whole script from my eRCaGuy_dotfiles repo here: repo_git_pull_latest.sh.

See also my cronjob readme and notes here: https://github.com/ElectricRCAircraftGuy/eRCaGuy_dotfiles/tree/master/useful_scripts/cronjobs