A production-ready database queue driver for Laravel

Updated: Jan 30, 2020 — 1 min Read#queues

The database queue driver was not recommended to run in production environments due to the problem of deadlocks with Relational database. Let's see how a deadlock happens:

Now we have a deadlock, multiple transactions waiting for each other to release the same locks.

MySQL 8.0.1 and Postgres 9.5 allows skipping a locked record and ignore it. This allowed us to prevents workers from fighting over the same job and thus cause a deadlock.

Starting the next Laravel release, you can use the queue database driver even if your environment is under heavy load. I've tested the changes processing 516,783 jobs by 50 workers and it resulted no deadlocks.

Pros and Cons of the database queue driver

The database driver is easy to setup since you don't need to worry about configuring and maintaining another tool "redis", it also opens the door for complex custom rules like funneling jobs per tenant or ensuring groups of jobs run in a certain order. In addition to this, you can easily configure database backups to ensure you don't lose the jobs in store in case the server crashed.

However, you should know that the database queue driver is not the best driver performance-wise, but it's still pretty efficient as the test results shared above implicate.

If you run MySQL >=8.0.1 or Postgres >=9.5, you don't need to change anything in your app. Just configure the database connection, run php artisan queue:table and php artisan migrate, and set the QUEUE_CONNECTION to database in your .env file.

If you want to read more about the new locking features in MySQL, check this post on the MySQL Server Blog.

By Mohamed Said

Hello! I'm a full-stack web developer working at Laravel. In this publication, I share everything I know about Laravel's core, packages, and tools.

You can find me on Twitter and Github.

Join the mailing list

If you like this article, you may want to consider joining the mailing list to receive new content once it is posted.
This site was built using Wink. Follow the RSS Feed.