Laravel Queues in Action (2nd edition) is now available!

Laravel Queues and Deployments

Updated: Feb 19, 2021 — 3 min Read#queues

When you deploy your application with new code or different configurations, workers running on your servers need to be informed about the changes. Since workers are long-living processes, they must be shut down and restarted in order for the changes to be reflected.

Restarting Workers Through The CLI

When writing your deployment script, you need to run the following command after pulling the new changes:

php artisan queue:restart

This command will send a signal to all running workers instructing them to exit after finishing any job in hand. This is called "graceful termination".

If you're using Laravel Forge, here's a typical deployment script that you may use:

cd /home/forge/
git pull origin master
$FORGE_COMPOSER install --no-interaction --prefer-dist --optimize-autoloader

( flock -w 10 9 || exit 1
    echo 'Restarting FPM...'; sudo -S service $FORGE_PHP_FPM reload ) 9>/tmp/fpmlock

$FORGE_PHP artisan migrate --force
$FORGE_PHP artisan queue:restart

Here the new code will be pulled from git, dependencies will be installed by composer, php-fpm will be restarted, migrations will run, and finally, the queue restart signal will be sent.

After php-fpm is restarted, your application visitors will start using the new code while the workers are still running on older code. Eventually, those workers will exit and be started again by Supervisor. The new worker processes will be running the new code.

If you're using Envoyer, then you need to add a deployment hook after the “Activate New Release” action and run the queue:restart command.

Restarting Workers Through Supervisor

If you have the worker processes managed by Supervisor, you can use the supervisorctl command-line tool to restart them:

supervisorctl restart group-name:*

Restarting Horizon

Similar to restarting regular worker processes, you can signal Horizon's master supervisor to terminate all worker processes by using the following command:

php artisan horizon:terminate

But in order to ensure your jobs won't be interrupted, you need to make sure of the following:

  1. Your Horizon supervisors' timeout value is greater than the number of seconds consumed by the longest-running job.
  2. Your job-specific timeout is shorter than the timeout value of the Horizon supervisor.
  3. If you're using the Supervisor process manager to monitor the Horizon process, make sure the value of stopwaitsecs is greater than the number of seconds consumed by the longest-running job.

With this correctly configured, Supervisor will wait for the Horizon process to terminate and won't force-terminate it after stopwaitsecs passes.

Horizon supervisors will also wait for the longest job to finish running and won't force-terminate after the timeout value passes.

Dealing With Migrations

When you send a restart signal to the workers, some of them may not restart right away; they'll wait for a job in hand to be processed before exiting.

If you are deploying new code along with migrations that'll change the database schema, workers that are still using the old code may fail in the middle of running their last job due to those changes; old code working with the new database schema!

To prevent this from happening, you'll need to signal the workers to exit and then wait for them. Only when all workers exit gracefully you can start your deployment.

To signal the workers to exit, you'll need to use supervisorctl stop in your deployment script. This command will block the execution of the script until all workers are shutdown:

sudo supervisorctl stop group-name:*

cd /home/forge/
# ...

$FORGE_PHP artisan migrate --force

sudo supervisorctl start group-name:*

{warning} Make sure the system user running the deployment can run the supervisorctl command as sudo.

Now, your workers will be signaled by Supervisor to stop after processing any jobs in hand. After all workers exit, the deployment script will continue as normal; migrations will run, and finally, the workers will be started again.

However, you should know that supervisorctl stop may take time to execute depending on how many workers you have and if any long-running job is being processed.

You don't want to stop the workers in this way if you don't have migrations that change the schema. So, I recommend that you don't include supervisorctl stop in your deployment script by default. Only include it when you know you're deploying a migration that will change the schema and cause workers running on old code to start throwing exceptions.

You can also manually run supervisorctl stop, wait for the command to execute, start the deployment, and finally run supervisorctl start after your code deploys.

If you want to learn more about Laravel's queue system, make sure to check Laravel Queues in Action! I've put everything I know about the queue system in an eBook format along with several real-life uses cases. Check it out for a crash course, a cookbook, a guide, and a reference.

Hey! 👋 If you find this content useful, consider sponsoring me on GitHub.

You can also follow me on Twitter, I regularly post about all things Laravel including my latest video tutorials and blog posts.

By Mohamed Said

Hello! I'm a former Laravel core team member & VP of Engineering at Foodics. In this publication, I share everything I know about Laravel's core, packages, and tools.

You can find me on Twitter and Github.

This site was built using Wink. Follow the RSS Feed.