The database queue driver was not recommended to run in production environments due to the problem of deadlocks with Relational database. Let's see how a deadlock happens:
- A worker puts a lock on a job to process it.
- Another worker finds the same job locked and it waits for the lock to be released.
- The first worker is done with the job and wants to delete it, so it requests another lock.
Now we have a deadlock, multiple transactions waiting for each other to release the same locks.
MySQL 8.0.1 and Postgres 9.5 allows skipping a locked record and ignore it. This allowed us to prevents workers from fighting over the same job and thus cause a deadlock.
Starting the next Laravel release, you can use the queue database driver even if your environment is under heavy load. I've tested the changes processing 516,783 jobs by 50 workers and it resulted no deadlocks.
Pros and Cons of the database queue driver
The database driver is easy to setup since you don't need to worry about configuring and maintaining another tool "redis", it also opens the door for complex custom rules like funneling jobs per tenant or ensuring groups of jobs run in a certain order. In addition to this, you can easily configure database backups to ensure you don't lose the jobs in store in case the server crashed.
However, you should know that the database queue driver is not the best driver performance-wise, but it's still pretty efficient as the test results shared above implicate.
If you run MySQL >=8.0.1 or Postgres >=9.5, you don't need to change anything in your app. Just configure the database connection, run php artisan queue:table and php artisan migrate, and set the QUEUE_CONNECTION to database in your .env file.
If you want to read more about the new locking features in MySQL, check this post on the MySQL Server Blog.
