When you deploy your application with new code or different configurations, workers running on your servers need to be informed about the changes. Since workers are long-living processes, they must be shut down and restarted in order for the changes to be reflected.
Restarting Workers Through The CLI
When writing your deployment script, you need to run the following command after pulling the new changes:
php artisan queue:restart
This command will send a signal to all running workers instructing them to exit after finishing any job in hand. This is called "graceful termination".
If you're using Laravel Forge, here's a typical deployment script that you may use:
cd /home/forge/mysite.com
git pull origin master
$FORGE_COMPOSER install --no-interaction --prefer-dist --optimize-autoloader
( flock -w 10 9 || exit 1
echo 'Restarting FPM...'; sudo -S service $FORGE_PHP_FPM reload ) 9>/tmp/fpmlock
$FORGE_PHP artisan migrate --force
$FORGE_PHP artisan queue:restart
Here the new code will be pulled from git, dependencies will be installed by composer, php-fpm will be restarted, migrations will run, and finally, the queue restart signal will be sent.
After php-fpm is restarted, your application visitors will start using the new code while the workers are still running on older code. Eventually, those workers will exit and be started again by Supervisor. The new worker processes will be running the new code.
If you're using Envoyer, then you need to add a deployment hook after the “Activate New Release” action and run the queue:restart command.
Restarting Workers Through Supervisor
If you have the worker processes managed by Supervisor, you can use the supervisorctl command-line tool to restart them:
supervisorctl restart group-name:*
Restarting Horizon
Similar to restarting regular worker processes, you can signal Horizon's master supervisor to terminate all worker processes by using the following command:
php artisan horizon:terminate
But in order to ensure your jobs won't be interrupted, you need to make sure of the following:
- Your Horizon supervisors'
timeoutvalue is greater than the number of seconds consumed by the longest-running job. - Your job-specific
timeoutis shorter than the timeout value of the Horizon supervisor. - If you're using the Supervisor process manager to monitor the Horizon process, make sure the value of
stopwaitsecsis greater than the number of seconds consumed by the longest-running job.
With this correctly configured, Supervisor will wait for the Horizon process to terminate and won't force-terminate it after stopwaitsecs passes.
Horizon supervisors will also wait for the longest job to finish running and won't force-terminate after the timeout value passes.
Dealing With Migrations
When you send a restart signal to the workers, some of them may not restart right away; they'll wait for a job in hand to be processed before exiting.
If you are deploying new code along with migrations that'll change the database schema, workers that are still using the old code may fail in the middle of running their last job due to those changes; old code working with the new database schema!
To prevent this from happening, you'll need to signal the workers to exit and then wait for them. Only when all workers exit gracefully you can start your deployment.
To signal the workers to exit, you'll need to use supervisorctl stop in your deployment script. This command will block the execution of the script until all workers are shutdown:
sudo supervisorctl stop group-name:*
cd /home/forge/mysite.com
# ...
$FORGE_PHP artisan migrate --force
sudo supervisorctl start group-name:*
{warning} Make sure the system user running the deployment can run the
supervisorctlcommand assudo.
Now, your workers will be signaled by Supervisor to stop after processing any jobs in hand. After all workers exit, the deployment script will continue as normal; migrations will run, and finally, the workers will be started again.
However, you should know that supervisorctl stop may take time to execute depending on how many workers you have and if any long-running job is being processed.
You don't want to stop the workers in this way if you don't have migrations that change the schema. So, I recommend that you don't include supervisorctl stop in your deployment script by default. Only include it when you know you're deploying a migration that will change the schema and cause workers running on old code to start throwing exceptions.
You can also manually run supervisorctl stop, wait for the command to execute, start the deployment, and finally run supervisorctl start after your code deploys.
If you want to learn more about Laravel's queue system, make sure to check Laravel Queues in Action! I've put everything I know about the queue system in an eBook format along with several real-life uses cases. Check it out for a crash course, a cookbook, a guide, and a reference.
