Contents

1 Laravel queues and jobs explained

Laravel queues and jobs explained

Friends, picture this: it's 2 AM, your app's humming along, but suddenly a user uploads a massive video. Processing it right there in the request? Disaster. The page hangs, users bounce, and you're left staring at a spinning loader, coffee going cold. That's the chaos Laravel queues save us from. They let you shove heavy lifting—emails, image resizing, API calls—into the background, keeping your app snappy and your sanity intact.

I've been there, knee-deep in a Laravel project where queues turned a bottlenecked mess into a smooth machine. Today, let's unpack this powerhouse feature: what it is, how it ticks, and how you can wield it like a pro. No fluff—just real talk from late-night debugging sessions.

Why queues matter in real projects

Queues aren't some abstract concept. They're your lifeline when synchronous code starts choking your app. Think about it: sending a welcome email during signup? Fine in dev, but in production with 10,000 users? Your server grinds to a halt.

I remember my first big e-commerce site. Order confirmations were timing out because PDF generation took 5 seconds each. Users thought the site was broken. Switched to queues, dispatched the PDFs async, and boom—lightning-fast checkouts. Response times dropped from 7s to 200ms.

Key wins:

Speed: Main thread stays free for user requests.
Scalability: Spin up workers on separate servers, handle spikes effortlessly.
Reliability: Jobs retry on failure, no lost data.
User experience: No more "please wait" screens that never end.

Have you ever watched a deployment tank because of a slow third-party API? Queues delegate that pain away.

The basics: Jobs, queues, and workers

At heart, a job is just a chunk of code you want to run later. Wrap it in a class, dispatch it to a queue (a holding pen), and let workers (background processes) chew through them.

Laravel makes this dead simple. No reinventing wheels.

Creating your first job

Fire up Artisan:

php artisan make:job SendWelcomeEmail

This spits out a class in app/Jobs. Here's a battle-tested example for processing user signups:

<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use App\Models\User;
use App\Mail\WelcomeEmail;
use Illuminate\Support\Facades\Mail;

class SendWelcomeEmail implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $tries = 3; // Retry up to 3 times
    public $timeout = 90; // Bail after 90 seconds

    public function __construct(public User $user) {}

    public function handle(): void
    {
        Mail::to($this->user->email)->send(new WelcomeEmail($this->user));
    }
}

Notice those traits? ShouldQueue marks it for the queue. Dispatchable handles the magic of firing it off.

Dispatch from a controller:

SendWelcomeEmail::dispatch($user);

Done. Email sends in the background while the user sees "Welcome aboard!" instantly.

Queue drivers: Pick your poison

Laravel supports a bunch: database (easiest start), Redis (blazing fast), SQS (AWS scale), Beanstalkd. Start with database for simplicity.

In .env:

QUEUE_CONNECTION=database

Run migrations:

php artisan queue:table
php artisan queue:failed-table
php artisan migrate

This creates jobs, failed_jobs, and job_batches tables. Jobs land in jobs, failures get tracked for retries.

Setting up and running workers

Workers are the muscle. They poll the queue, grab jobs, execute them, delete on success.

Spin one up:

php artisan queue:work

That's it. For production, use Supervisor to keep them alive. Multiple queues? php artisan queue:work --queue=high,default.

I once had a queue backlog spike to 50k during Black Friday. Workers on Redis chewed it down in hours. Database queues would've choked.

Diving deeper: Priorities, failures, and pro tips

Queues get sophisticated fast. You've got multiple queues for prioritization—like "high" for payments, "low" for analytics.

Dispatch to a specific one:

ProcessImage::dispatch($image)->onQueue('images');

Workers prioritize: queue:work --queue=high,images,default.

Handling the ugly: Failed jobs

Shit happens. Network blips, API downtime. Laravel tracks failures in failed_jobs.

Retry 'em:

php artisan queue:retry all
# Or specific ID
php artisan queue:retry 5

Set job-level retries in the class ($tries = 5), or use exponential backoff:

public $backoff = [10, 30, 60]; // Delays between tries

Pro move: Custom failure handling.

public function failed(\Throwable $exception): void
{
    // Log to Slack, notify admin
    Log::error('Job failed: ' . $exception->getMessage());
}

Middleware: Rate limiting, throttling, and more

Jobs support middleware—like guards before execution. Perfect for APIs with limits.

Example rate limiter:

<?php

namespace App\Jobs\Middleware;

use Illuminate\Cache\RateLimiter;
use Illuminate\Support\InteractsWithTime;

class RateLimited
{
    use InteractsWithTime;

    public function __construct(
        protected string $key,
        protected int $maxAttempts = 10,
        protected int $decaySeconds = 60
    ) {}

    public function handle($job, $next)
    {
        $key = $this->key($job);

        if (RateLimiter::tooManyAttempts($key, $maxAttempts)) {
            return $job->release($this->availableIn($key));
        }

        RateLimiter::hit($key, $decaySeconds);

        return $next($job);
    }

    protected function key($job): string
    {
        return 'jobs:'.$job->uuid();
    }

    public function availableIn($key): int
    {
        return RateLimiter::availableIn($key);
    }
}

Slap it on a job:

class ApiCaller implements ShouldQueue
{
    public function middleware()
    {
        return [new RateLimited('api-calls')];
    }
}

This saved my ass on a Twilio integration—prevented bans during spikes.

Batches: Orchestrating job symphonies

Laravel 8+ batches let you chain jobs. Render 100 images? Batch 'em, track progress, chain notifications.

$batch = Bus::batch([
    new ProcessImage($image1),
    new ProcessImage($image2),
])->then(function (Batch $batch) {
    // All done!
    Notification::send($user, new ImagesProcessed());
})->dispatch();

Catch then, catch, finally callbacks. Progress tracking via $batch->progress().

Real-world: User uploads podcast batch. Queue processing, notify when ready.

Advanced: Customizing connections and events

Hook into events for logging:

use Illuminate\Support\Facades\Queue;
use Illuminate\Queue\Events\JobFailed;

Queue::failing(function (JobFailed $event) {
    // Slack alert
});

Multiple connections in config/queue.php:

'connections' => [
    'redis-high' => ['driver' => 'redis', 'queue' => 'high'],
    'database' => ['driver' => 'database'],
],

Dispatch to connection: ProcessImage::dispatch($image)->onConnection('redis-high');

Production war stories and pitfalls

Deployed queues? Here's what bites newbies.

Pitfall 1: Visibility timeout. Worker "locks" a job for, say, 60s. Job runs 70s? Duplicates. Set it higher than your longest job.

Pitfall 2: Sync driver in prod. .env says sync? Jobs run inline. Disaster.

Pitfall 3: No monitoring. Use Horizon for Redis (gorgeous dashboard) or Telescope. Watch backlogs like a hawk.

My nightmare: Forgot to migrate failed_jobs on staging. Failures vanished into ether. Always test the full flow.

Scaling tips:

Redis + multiple workers per server.
Supervisor config for restarts.
Separate DB connection for queues to avoid locks.
Prune old failed jobs: php artisan queue:prune-failed --hours=48.

For massive scale, SQS or RabbitMQ shine, but Redis handles most apps fine.

Putting it all together: Image processor example

Let's build something real. User uploads image—queue thumbnails, optimize, notify.

Job class:

<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Intervention\Image\Facades\Image;
use App\Models\ImageUpload;

class ProcessImageUpload implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $tries = 3;
    public $timeout = 120;

    public function __construct(public ImageUpload $image) {}

    public function handle(): void
    {
        $sizes = [150, 300, 600];

        foreach ($sizes as $size) {
            $thumb = Image::make($this->image->path)
                ->resize($size, $size, function ($constraint) {
                    $constraint->aspectRatio();
                    $constraint->upsize();
                })
                ->save(storage_path("thumbnails/{$size}_{$this->image->filename}"), 85);
        }

        // Optimize original
        Image::make($this->image->path)->encode('jpg', 90)->save($this->image->path);

        $this->image->update(['status' => 'processed']);
    }
}

Controller dispatch:

public function store(Request $request)
{
    $image = ImageUpload::create([...]);
    ProcessImageUpload::dispatch($image)->onQueue('images');
    return response()->json(['message' => 'Processing started!']);
}

Worker: php artisan queue:work --queue=images

Users get instant feedback, images ready minutes later. Magic.

Colleagues, queues transformed how I build. They whisper reliability into frantic deadlines, turning "hold on" into "done." Next time your app feels sluggish, dispatch that pain away—feel the freedom linger.

Transform Your Laravel App: How Queues and Jobs Revolutionize Performance and User Experience