How to Create Reliable Scheduled Tasks in PHP That Never Run Twice and Avoid Costly Mistakes

Contents

1 Creating reliable scheduled tasks without duplicates: stories from the trenches
2 The quiet problem: when tasks run twice
3 Why overlapping happens more often than we admit
4 First line of defense: simple locks that actually work
5 PHP-land tricks: when logic lives in your code
- 5.1 1. Database-based coordination
- 5.2 2. Advisory locks (MySQL / PostgreSQL)
6 Laravel world: withoutOverlapping() and friends
7 The deeper layer: design with idempotency in mind
- 7.1 Examples of idempotent patterns
- 7.2 The mental shift
8 Real-world scenario: end-of-month billing
9 Subtle traps: when prevention looks correct but isn’t
10 Practical patterns you can start using today
11 The human side of reliable tasks

Creating reliable scheduled tasks without duplicates: stories from the trenches

There is a specific kind of quiet dread that only backend developers know.

It’s 7:42 in the morning. You’re half awake, staring at the monitoring dashboard. Somewhere in the background, your coffee machine grumbles. And there it is:
yesterday’s “harmless” cron job just duplicated 40,000 invoices.

No exception. No red error. Just a silent, polite disaster.

If you work with PHP cron jobs, Laravel Scheduler, or any custom queue system long enough, you either already lived this, or you’re on your way.

Let’s talk about that.

Not just “how to avoid overlapping cron jobs” in PHP, but how to design reliable scheduled tasks that do not break when the world is slightly imperfect. Because it will be.

And because you and I both know: “but this script only runs once a day” is not a safety guarantee. It’s wishful thinking.

The quiet problem: when tasks run twice

Have you ever seen this kind of pattern?

Cron: * * * * * php artisan schedule:run
Task: “Generate yesterday’s report”
Production server under load
Script that usually runs in 10 seconds suddenly runs in 80
Before it finishes, the next minute ticks over
Scheduler starts another instance

You don’t even need Laravel for this. Plain PHP + cron is enough:

You set */5 * * * * php /var/www/app/cron/report.php
One day the database is slow, or remote API times out
First run is still working when the second one starts
Both scripts write to the same tables, send the same emails, charge the same customers

The overlap is invisible until someone from finance pings you:

“Why did customer X get charged twice?”

Why overlapping happens more often than we admit

Some reasons are technical, some are just… human.

Cron doesn’t care about your runtime. It fires on schedule, not on “when the previous job finished.”
Servers are not constant. That 10-second job becomes 3 minutes when backups, CPU bursts, and slow I/O join the party.
Distributed deployments. Two PHP-FPM instances, a couple of workers, maybe horizontal scaling. Suddenly “only one script at a time” becomes a myth.
Naive assumptions. We rely on “this job is fast” instead of treating tasks as critical, state-changing code.

And in PHP, it’s especially common: many of us start with “just add a cron line” and move on. Until some night, something overlapping quietly unfolds your database like a cheap lawn chair.

First line of defense: simple locks that actually work

The most basic idea: before a job does anything, it claims a lock. If the lock is already taken, it bails out.

You can do this in a bunch of ways:

1. Lock files

The classic, old-school, surprisingly effective method.

At the start of your script, you check for a lock file (e.g. /tmp/report.lock).
If it exists and is still valid → exit.
If not → create it, run, delete it at the end.

Something like:

<?php

$lockFile = sys_get_temp_dir() . '/daily_report.lock';

if (file_exists($lockFile)) {
    // Someone else is running this job
    exit(0);
}

file_put_contents($lockFile, getmypid());

try {
    // ... your critical job: generate report, send emails, etc.
} finally {
    if (file_exists($lockFile)) {
        unlink($lockFile);
    }
}

This is fine for:

Single server
Simple cron tasks
Low-risk jobs

But there are pitfalls:

If the process dies mid-run (fatal error, OOM, server crash), the lock file might stay forever.
You might want to store a PID + timestamp in it, and consider “expired locks” if they’re too old.
It doesn’t protect you across multiple machines unless they share the same filesystem.

Still, for many PHP cron jobs on a single server, a lock file is the difference between “once per day” and “randomly twice because the disk was slow.”

2. `flock` from the outside

Sometimes you don’t even need to touch PHP code. You can wrap the job at the cron level using flock:

* * * * * /usr/bin/flock -n /tmp/daily_report.lockfile php /var/www/app/cron/report.php

flock takes a lock on /tmp/daily_report.lockfile.
If another cron instance is already holding the lock, this one exits immediately (-n = non-blocking).
Your script stays simple; the locking is handled by the shell.

This trick is especially nice when:

You inherited a legacy PHP script you don’t want to touch too much.
You’re deploying on Linux and have control over the crontab.
You want a minimalistic, battle-tested approach.

3. PID files and process checks

Some teams prefer to track the PID explicitly:

Write getmypid() into /var/run/yourjob.pid.
Next run checks:
- if PID file exists
- and a process with that PID is actually running

If yes → exit. If not → take over and update the PID.

This is more complex than a simple file existence check, but it helps debugging:

“Which process is running this job?”
“Is that process stuck?”

You can inspect the PID from the server directly.

PHP-land tricks: when logic lives in your code

Let’s say you’re not just calling a single script, but orchestrating several things from PHP:

dispatching workers
reading from job tables
using some homegrown queue

There, you’ll want a bit more structure.

1. Database-based coordination

A surprisingly robust pattern: let the database guarantee uniqueness.

Example: simple “jobs” table:

CREATE TABLE jobs (
    id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
    type VARCHAR(50),
    status ENUM('pending','processing','done') NOT NULL DEFAULT 'pending',
    worker_id VARCHAR(50) NULL,
    run_at DATETIME NOT NULL,
    UNIQUE KEY unique_daily_report (type, run_at)
);

And in PHP:

$workerId = gethostname() . ':' . getmypid();

// Try to claim one job
$updated = $pdo->prepare("
    UPDATE jobs
    SET status = 'processing', worker_id = :worker
    WHERE status = 'pending'
      AND run_at <= NOW()
    ORDER BY id
    LIMIT 1
")->execute(['worker' => $workerId]);

if (!$updated) {
    // No job claimed – maybe another worker took it
    exit(0);
}

// Now select the job we own
$stmt = $pdo->prepare("SELECT * FROM jobs WHERE worker_id = :worker AND status = 'processing'");
$stmt->execute(['worker' => $workerId]);
$job = $stmt->fetch(PDO::FETCH_ASSOC);

if (!$job) {
    exit(0);
}

// ... process the job ...

$pdo->prepare("
    UPDATE jobs
    SET status = 'done', worker_id = NULL
    WHERE id = :id
")->execute(['id' => $job['id']]);

The database is your lock:

Only one UPDATE wins (because it updates based on status='pending').
Others get 0 rows updated and know they lost the race.
You can scale horizontally without tasks stepping on each other.

2. Advisory locks (MySQL / PostgreSQL)

Most relational databases have some notion of advisory locks:

MySQL: GET_LOCK('job-name', timeout)
PostgreSQL: pg_advisory_lock(key)

Within PHP, you can:

Ask the DB for a named lock.
If you get it → run the job.
If not → exit.

The nice part: no files, no weird global state, and locks are usually scoped to the connection and released if the connection drops.

It works especially well when:

You’re already hitting the database inside the job anyway.
You want a global, distributed lock across multiple PHP workers.

Laravel world: `withoutOverlapping()` and friends

If your world is Laravel, you already have some powerful tools built in.

You’ve probably seen this:

$schedule->command('report:generate')
    ->dailyAt('01:00')
    ->withoutOverlapping();

Under the hood:

Laravel acquires a mutex (a cache-based lock) when the command starts.
The next time the scheduler tries to run it, it checks that lock.
If the lock exists → command is skipped.

The deeper layer: design with idempotency in mind

Now, here’s the uncomfortable truth:

Even with the best lock file, perfect cron, database advisory locks, Laravel mutexes — duplicate executions can still happen.

Network partitions
Cache outages
Process killed just after acquiring lock but before writing state
Human mistakes in deploys

So the real question is not just: “How do I prevent overlap?”

It’s also:

If my job runs twice, what keeps it from hurting people?

This is where idempotency comes in.

Idempotency means:
Running the same operation multiple times leads to the same end state.

Examples of idempotent patterns

Before sending a payment request, you generate a unique request ID and store it.
- When the external API calls back, you check that ID and make sure you don’t process it twice.
When creating a daily report, you store it at a deterministic path:
- /reports/2026-07-20-summary.pdf
- If it exists, you overwrite or skip, not create a new one.
When inserting rows, you use unique indexes:
- e.g. UNIQUE (user_id, date) so inserting the same job twice fails gracefully.
When processing webhooks, your handler can safely handle re-delivery:
- It checks a table of processed event IDs.
- If the event is already processed, it skips.

So then — even in that bad day where everything overlaps and runs twice — the system might:

do a bit more work,
log more stuff,
but not double-charge, not double-send, not double-insert.

The mental shift

When I design scheduled tasks now, I ask this question early:

“If this command runs twice at the same time, what stops real-world damage?”

And I expect multiple layers to answer:

A lock (file, flock, DB, Redis, Laravel mutex)
A uniqueness boundary (DB constraint, external id, deterministic path)
A safe logic path (check-before-insert, check-before-send)

Real-world scenario: end-of-month billing

Let’s make it concrete.

You have a PHP job:

Runs on the last day of the month
Reads all active subscriptions
Generates invoices
Charges cards

At first, you might just write:

foreach ($subscriptions as $subscription) {
    $invoiceId = createInvoice($subscription);
    chargeCard($subscription->card, $invoiceId);
}

And wrap it with a cron entry.

Then one month, due to some race condition or manual job re-run, it executes twice. Users get two invoices, two charges.

Now imagine the same scenario designed with constraints:

Your invoices table has a unique constraint:

UNIQUE KEY unique_subscription_period (subscription_id, billing_period_start, billing_period_end);

Your billing logic checks if an invoice for that period exists before creating one.
You generate a idempotency key per subscription + period and store it when charging.

Now, even if:

the job is started twice by mistake, or
your lock fails for once in its life,

you end up with:

one invoice per period
one charge per invoice
potential duplicate log noise
but no broken customer trust

Subtle traps: when prevention looks correct but isn’t

It’s easy to feel safe while still sitting on a mine.

Some common gotchas:

1. “The script is short, it can’t overlap”

You think: “This only takes 2 seconds.”
Production thinks: “Hold my beer.”

Backup job spikes I/O.
DB query that used to take milliseconds now takes 30 seconds.
Remote API rate-limits you.

Avoid locking based on optimistic runtime. Measure worst-case. Add margin. Then design as if it still may go wrong.

2. Per-server locks in a distributed setup

You use a file lock or file cache on each app server:

Server A gets a lock; runs the job.
Server B gets its own lock; also runs the job.

You think “I have a lock.”
You have two locks in two separate universes.

If your job is meant to run only once globally:

Put your lock in something that all servers see:
- shared filesystem
- MySQL, PostgreSQL
- Redis
- Laravel cache backed by shared storage

3. Over-long lock expiry

In Laravel:

->withoutOverlapping()

with default settings often means: lock expires after 24 hours.

If your job usually takes 1 minute, and it crashes 10 seconds in, that lock will still sit there all day.

You think: “The scheduler is broken.”
In reality, the lock is faithfully doing its job, but with unrealistic assumptions.

Better:

Pick withoutOverlapping(10) for a job that should finish under 10 minutes.
That way, if it dies, you only lose at most 10 minutes before it can recover with a new run.

4. Over-ideal tests

We run scheduled tasks in local/dev with:

tiny datasets
no real network latency
no competing workloads

Everything looks perfect. Then production adds:

real users
spikes
background jobs
scheduled backups
index rebuilds
cache flushes

Testing time-based behavior under realistic load is hard. But even just consciously thinking about “how would this behave if everything were 10x slower?” reveals edge cases that need locking and idempotency.

Practical patterns you can start using today

Let’s put the ideas into a small checklist you can adapt in your own PHP projects, Laravel or not.

For classic PHP + cron

Use flock in your crontab:

* * * * * /usr/bin/flock -n /tmp/somejob.lockfile php /var/www/app/cron/somejob.php

Or implement lock files inside PHP with PID + timestamp.
For more safety on shared infra, move to database advisory locks or a jobs table model where the DB coordinates ownership.

For Laravel scheduler

For tasks longer than a blink:

$schedule->command('report:generate')
    ->everyFiveMinutes()
    ->withoutOverlapping(15)
    ->onOneServer();

Ensure your CACHE_DRIVER is shared across nodes if you use onOneServer.
Make underlying operations idempotent:
- unique keys in DB
- deterministic file paths for generated artifacts
- idempotency keys for external APIs

For any environment

Ask these questions while reviewing or writing scheduled jobs:

What happens if this runs twice at the same time?
What if it runs twice in sequence due to a manual re-run?
What state changes are truly irreversible?
Where is the uniqueness enforced — in code, DB, or not at all?

And answer them not only with “we have a lock,” but with multiple layers:

Locking
Unique constraints
Idempotent logic
Monitoring and clear logs

The human side of reliable tasks

There’s one more angle I think about a lot, and I suspect you do too.

Behind each cron job, behind each Laravel command, there is a person who will be paged at 3 AM if it goes wrong. Sometimes that person is you. Sometimes it’s the next developer who takes over the project.

Reliable scheduled tasks are not just about correctness. They are about kindness:

Kindness to the person debugging a production incident.
Kindness to the team that has to maintain this system next year.
Kindness to users who never see your code, but feel the impact of double emails, double invoices, missing reports.

When you add a small withoutOverlapping(30) or a careful unique index, you’re not just writing “enterprise best practice.” You’re quietly leaving a gift to a future colleague who might be staring at the logs on a tired morning, grateful that, at least this part didn’t explode.

And maybe, one late night, when the monitor glow is the only light in the room and you’re scrolling through a job’s logs, you’ll notice that something could have gone wrong — but didn’t — because you took the time to design for it.

Those small, invisible wins rarely make it into release notes.
But they’re the ones that let us breathe a little easier and keep moving forward.