PHP long-running scripts explained

Hey, fellow developers. Picture this: it's 2 AM, your keyboard's the only light in the room besides the glow of your monitor. You've got a massive dataset—millions of rows from some client import—that needs crunching. One script. Fire it up, lean back, and… timeout. PHP's default limits hit like a brick wall. Frustrating, right? We've all been there, staring at that "Maximum execution time exceeded" error, wondering why PHP, our trusty workhorse, suddenly feels so fragile.

Long-running PHP scripts aren't just a nuisance; they're a reality in data processing, queue workers, API streams, or nightly batch jobs. PHP wasn't built for marathons—it thrives on quick hits, like serving web pages. But with tweaks, patterns, and a bit of grit, you can make it run for hours, days even, without crashing your server or your sanity. Let's dive in, share some battle-tested approaches, and reflect on why this matters for us PHP folks chasing reliability in a world of flaky deploys.

Why PHP resists the long haul

PHP's DNA screams "short-lived." Defaults like max_execution_time=30 and memory_limit=128M keep things snappy for HTTP requests. Push beyond, and you invite memory leaks, garbage collection hiccups, or zombie processes eating your RAM.

I remember my first big one: processing Twitter streams for a sentiment analysis tool. Script ran fine for 20 minutes, then poof—memory ballooned to 2GB. Why? Globals holding onto arrays, no cleanup. PHP's garbage collector isn't magic; it struggles with circular references in long loops. The lesson? Tune or die trying.

Key pitfalls:

• Memory creep: Iterating huge arrays loads everything at once.
• No built-in restarts: Crash once, and your job's toast.
• Web server timeouts: Even with set_time_limit(0), Apache/Nginx cut you off after 120 seconds tops.

But here's the spark: PHP can endure. It's about smart design, not brute force.

Batch it: The AJAX progress bar savior

One evening, knee-deep in a CSV import for a WordPress plugin, my monolithic script choked after 10,000 rows. Solution? Break it into batches. Process 200 rows, report progress, repeat via AJAX. No php.ini tweaks, no server hangs. Users see a smooth bar filling up—no blank screens.

Here's the magic from that plugin I hacked together. Frontend: a simple form with a "current_row" hidden field starting at 0.

<form method="post" action="" id="script-form" enctype="multipart/form-data">
    <input type="hidden" name="action" value="run_script">
    <input type="hidden" name="current_row" value="0">
    <input type="hidden" name="ajax-nonce" value="<?php echo wp_create_nonce('ajax-nonce'); ?>">
    <button id="script-submit">Run Script</button>
</form>

JavaScript loops it recursively:

function processScript(formData) {
    $.getJSON(ajaxurl, formData)
        .done(function(data) {
            if (data.result === 'COMPLETE') {
                alert('Done!');
            } else if (data.result === 'NEXT') {
                $('input[name="current_row"]').val(data.next_row);
                processScript($('form#script-form').serialize());
            } else {
                alert('Fail!');
            }
        });
}

PHP side: Open the CSV, seek to current_row, process block_size=200 rows, count totals with transients for persistence. Return 'NEXT' with updated row pointer, or 'COMPLETE' when done. Nonce checks and admin-only access keep it secure.

This scales beautifully. Tweak block size for your server's appetite—too big, memory spikes; too small, too many requests. Pro tip: Add a progress bar by calculating current_row / total_rows * 100.

Have you tried this on a Laravel queue? Same idea: jobs in chunks, with Horizon for monitoring.

Daemon dreams: Cron, nohup, and infinite loops

Batch via web? Fine for interactive jobs. But for background beasts—like nightly data crunches or Twitter streams—you need detachment. Enter cron jobs, nohup, and bash wrappers. PHP shines here if you let it "die gracefully" and restart.

At Review Signal, they crunch millions of data points nightly. Bash script launches PHP, cron fires it at 11 PM. It runs till done, no matter the hours.

Simple bash launcher:

#!/bin/sh
php /path/to/your_script.php

Cron: 0 23 * * * /path/to/launcher.sh

Lose SSH? nohup php script.php & keeps it alive. Check with jobs or ps aux | grep script.php.

For always-on, like streaming APIs: a "forever" script.

#!/bin/sh
while true; do
    ps aux | grep '[y]our_script.php' || php /path/to/your_script.php
    sleep 10
done

Cron every minute: * * * * * /path/to/forever.sh. If process dies, it respawns. Add exponential backoff for failures—random delays up to 5 minutes prevent thundering herds.

PHP internals for daemons? Use pcntl_signal for graceful SIGTERM shutdowns:

<?php
pcntl_signal(SIGTERM, function() { exit(0); });
while (!$shouldStop()) {
    $item = get_next_item(); // Cursor-style, low memory
    process($item);
    commit($item);
    if (memory_get_usage(true) > 1e9) exit(0); // Self-restart at 1GB
}

Set memory_limit=-1, max_execution_time=0 in php.ini or CLI. Avoid globals/statics—they leak. Cursor iteration (e.g., yield generators in PHP 7+) keeps memory flat.

Memory mastery and PHP's hidden traps

Long-runners kill servers via leaks. Tune with Tideways: profile garbage collection, spot retainers. Rules:

• Generators over arrays: foreach (new Generator() as $item)
• Close DB connections: $pdo = null;
• No singletons in loops.

Exponential backoff on restarts: Fail once? Wait 1s. Twice? 2s. Caps at 300s. Random jitter avoids pile-ups.

Laravel queues? php artisan queue:work --timeout=0 with Supervisor (if allowed). No Supervisor? That forever bash loop.

Wappler folks cap at 120s per call—queue via webhooks + cron for daily batches.

Real-world scars and quiet wins

I once let a script run six months on AWS micro—20GB data, no restarts. RDS filled first. PHP held up, but sloppy code would've leaked. Another time, a bash loop caught a segfault from buggy GC; restarted seamlessly.

Tradeoffs table for your toolkit:

Test ruthlessly: ab -n 1000 for loads, valgrind for leaks (if compiled).

Friends, long-running PHP isn't about fighting the language—it's partnering with it. Write lean, restart smart, monitor fiercely. Next time that 2 AM glow hits, you'll smile knowing your script's humming, not dying.

That quiet confidence? It's what keeps us coding through the night.