Monitoring Cron Jobs

VCU TS Healthchecks is perfectly suited for monitoring cron jobs. All you have to do is update your cron job command to send a HTTP request to VCU TS Healthchecks after a job completes.

Let's look at an example:

$ crontab -l
# m h dom mon dow command
  8 6 * * * /home/user/backup.sh

The above job runs /home/user/backup.sh every day at 6:08. The backup script is presumably a headless, background process. Even if it works correctly currently, it can start silently failing in future, without anyone noticing.

You can set up VCU TS Healthchecks to notify you whenever the backup script does not run on time or does not complete successfully. Here are the steps to do that.

  1. If you have not already, sign up for a free VCU TS Healthchecks account.

  2. In your VCU TS Healthchecks account, add a new check.

  3. Give the check a meaningful name. Good naming will become increasingly important as you add more checks to your account.

  4. Edit the check's schedule:

    • change its type from "Simple" to "Cron"
    • enter 8 6 * * * in the cron epression field
    • set the timezone to match your machine's timezone

  5. Take note of your check's unique ping URL

Finally, edit your cron job definition and append a curl or wget call after the command:

$ crontab -e
# m h dom mon dow command
  8 6 * * * /home/user/backup.sh && curl -fsS --retry 3 https://healthcheck.ts.vcu.edu/ping/your-uuid-here > /dev/null

Now, each time your cron job runs, it will send a HTTP request to the ping URL. Since VCU TS Healthchecks knows the schedule of your cron job, it can calculate the dates and times when the job should run. As soon as your cron job doesn't report at an expected time, VCU TS Healthchecks will send you a notification.

This monitoring technique takes care of various failure scenarios that could potentially go unnoticed otherwise:

  • The whole machine goes down (power outage, janitor stumbles on wires, VPS provider problems, etc.)
  • cron daemon is not running, or has invalid configuration
  • cron does start your task, but the task exits with non-zero exit code

Curl Options

The extra options tells curl to not print anything to standard output unless there is an error. Feel free to adjust the curl options to suit your needs.

&& Run curl only if /home/user/backup.sh exits with an exit code 0
-f, --fail Makes curl treat non-200 responses as errors
-s, --silent Silent or quiet mode. Don't show progress meter or error messages.
-S, --show-error When used with -s it makes curl show error message if it fails.
--retry <num> If a transient error is returned when curl tries to perform a transfer, it will retry this number of times before giving up. Setting the number to 0 makes curl do no retries (which is the default). Transient error means either: a timeout, an FTP 4xx response code or an HTTP 5xx response code.
> /dev/null Redirect curl's stdout to /dev/null (error messages go to stderr,)

Looking up Your Machine's Time Zone

On modern GNU/Linux systems, you can look up the time zone using the timedatectl status command and looking for "Time zone" in its output:

$ timedatectl status

               Local time: C  2020-01-23 12:35:50 EET
           Universal time: C  2020-01-23 10:35:50 UTC
                 RTC time: C  2020-01-23 10:35:50
                Time zone: Europe/Riga (EET, +0200)
System clock synchronized: yes
              NTP service: active
          RTC in local TZ: no