Laravel Queues in Action is now available!

Avoiding Memory Leaks When Running Laravel Queue Workers

Updated: Sep 21, 2020 — 1 min Read#queues

There has been a debate—for a long time—on whether or not PHP is a good choice for long-running processes. Based on my experience working on large scale projects that utilize hundreds of workers, I'm on the side of the debate that believes PHP is very efficient for getting the right job done.

The job of our workers here is running a Laravel application—PHP application—and get it to process background jobs. I've seen how powerful Laravel workers are with crunching a very large number of jobs on different server capacities and doing the job pretty well.

With that being said, avoiding memory leaks can still be a bit challenging. Over time, some references will pile up in the server memory that won't be detected by PHP and will cause the server to crash at some point.

The solution is easy though, restart the workers more often.

👋 This post is part of the "Laravel Queues in Action" eBook. Check it out for a crash course, a cookbook, a guide, and a reference.

With a process manager like Supervisor in place, you can restart your workers every hour to clean the memory knowing that Supervisor will start them back for you automatically.

To do this, you can configure a CRON job to run every hour and call the queue:restart command:

0 * * * * forge php /home/forge/ queue:restart

Adding this to your /etc/crontab file will run queue:restart every hour. Workers will receive that signal and exit after finishing any job in hand.

If you don't want to use a CRON task, you can use the --max-jobs and --max-time options to limit the number of jobs the worker may process or the time it should stay up:

php artisan queue:work --max-jobs=1000 --max-time=3600

This worker will automatically exit after processing 1000 jobs or after running for an hour. After processing each job, the worker will check if it exceeded max-jobs or max-time and then decide between exiting or picking up more jobs. The worker will not exit while in the middle of processing a job.

Finally, you may signal the worker to exit from within a job handle method:

public function handle()
    // Run the job logic.

    app('queue.worker')->shouldQuit  = 1;

This method can be useful if you know this specific type of job could be memory consuming and you want to make sure the worker restarts after processing it to free any reserved resources.

Using Horizon

When using Laravel Horizon, you can restart the Horizon process using a CRON job or you can configure maxJobs and maxTime in your Horizon supervisor configurations:

'environments' => [
    'production' => [
        'supervisor-1' => [
            // ...
            'maxTime' => 3600,
            'maxJobs' => 1000

Occasionally restarting your workers is a very efficient strategy for dealing with memory leaks.

Hey! 👋 If you want to receive updates on what I'm up to, I host a newsletter on my website and would love to have you.

You can also follow me on Twitter, I regularly post about all things Laravel including my latest video tutorials and blog posts.

By Mohamed Said

Hello! I'm a full-stack web developer working at Laravel. In this publication, I share everything I know about Laravel's core, packages, and tools.

You can find me on Twitter and Github.

This site was built using Wink. Follow the RSS Feed.