Scrapeless Wiki

What Is a Cron Job? – Definition and How It Works

Informational P2 what is a cron job

Learn what cron is, how the schedule expression works, the failure modes that bite in production, and when a scheduler is a better fit.

Cron runs commands on a schedule. It is the oldest and most widely available job scheduler on Unix-like systems, and it is where most recurring collection starts.

1. The Schedule Expression

β”Œβ”€ minute (0-59)
β”‚ β”Œβ”€ hour (0-23)
β”‚ β”‚ β”Œβ”€ day of month (1-31)
β”‚ β”‚ β”‚ β”Œβ”€ month (1-12)
β”‚ β”‚ β”‚ β”‚ β”Œβ”€ day of week (0-6, Sunday = 0)
β”‚ β”‚ β”‚ β”‚ β”‚
0 3 * * *   /usr/local/bin/collect.sh

That entry runs daily at 03:00. Ranges, lists, and step values (*/15 for every fifteen minutes) cover most needs.

2. The Environment Trap

Cron runs with a minimal environment β€” a short PATH, no shell profile, and often a different working directory.

A script that works in a terminal and fails under cron is almost always this. The fix is to use absolute paths, set required variables explicitly in the job or the script, and avoid depending on interactive shell configuration.

3. Output Disappears

By default cron mails output, and on most systems mail is not configured, so output vanishes. A job can fail silently for months.

Always redirect output to a log file, and check exit status:

0 3 * * * /usr/local/bin/collect.sh >> /var/log/collect.log 2>&1

4. Overlapping Runs

Cron starts a job on schedule regardless of whether the previous run finished. A collection taking longer than its interval eventually runs several copies at once, which multiplies load on the destination and can corrupt shared state.

A lock file or flock prevents it, and is worth adding before it becomes a problem.

5. Time Zones and Daylight Saving

Cron uses the system time zone. At a daylight saving transition, jobs scheduled in the skipped hour may not run, and jobs in the repeated hour may run twice.

For anything sensitive, schedule in UTC and convert inside the job.

6. What Cron Does Not Provide

Need Cron A scheduler
Retries No Yes
Dependencies between jobs No Yes
Backfills No Yes
Run history and visibility No Yes
Alerting on failure No Yes
Distributed execution No Yes

Those gaps are why pipelines eventually move to a dedicated scheduler.

7. When Cron Is Still Right

A single independent job, on one machine, with no dependencies, whose failure is tolerable until the next run. That describes a great deal of useful work, and adding an orchestration platform to it is overhead.

Add monitoring β€” a heartbeat the job pings on success β€” and simple cron covers a lot of ground.

8. Summary

Cron runs commands on a five-field schedule, with a minimal environment, no retries, no dependency handling, and output that disappears unless redirected.

Use absolute paths, log explicitly, prevent overlap with a lock, schedule in UTC, and monitor success β€” and move to a real scheduler once jobs depend on each other or need backfills.