It's never too late to share something useful with mates. And in this blog, I am going to discuss Queue Worker/API and how to implement it in Drupal 8. After completing this blog cum tutorial you will find yourself at a moderate level of expertise in implementing Queue API.
A couple of months back, we had a situation in one of our project where we had to pull data from a centralized server and update that content to the site almost every day. In order to overcome this scenario, we started using cron, but faced a situation where we were missing few contents because of time out. After a little research and discussion, we (our team) decided to implement queue worker. And yes! It worked.
Let’s start with Queue API.
Queue API enables us to handle a number of tasks at a later stage that means we can insert a number of items in the queue and process them later. The items can be added to the queue by passing an arbitrary data object. And we can run Queue by manually using Drush commands or Cron. Cron is a background process that runs at periodic intervals of time. For an introduction to cron, you can check out our previous blog on “scheduling Automated Tasks in Drupal with Cron”.
So why should we use Queue Worker?
Queue Worker allows you to run queue with time limits. You can also individually queue the tasks at one time compared to Cron that run at the periodic interval of time. In comparison to cron, Queue is more efficient and can handle resource-intensive tasks. The API also allows you to revert the item back to queue if any failure occurs. Most importantly, you can run multiple queues without interrupting other work.
The very basic concept of Queue API is FIFO (First-In-First-Out).
Check out an example of the above-mentioned scenario.
Here we will create a Queue Worker that will fetch data from centralized repo using API and create an article on our existing site. This process will run automatically based on cron.
See the screenshot below for an example of the folder structure that will contain required files and their respective location:
Module Setup:
article_queue.info.yml
In Drupal 8, the menu system has been replaced by the routing system.
article_queue.routing.yml
ArticleSettingsForm.php
Create the Configuration form to store API link and their credentials. After entering Connector Site URL, Site User Name and Site Password, save the configuration.
Let's see how we can use cron to add data to Queue.
In a hook_cron method, we will be defining a method to fetch data from API and adding to queue.
First, create a QueueFactory object from the service container statically and use this object to instantiate a queue object of article_queue_processor.
Here we call create Item method to insert items in the queue. Once cron runs it will insert all items fetched from API in the process queue.
article_queue.module
In this example, we will be using ultimate cron for processing the cron. We need to set up we need to create a config file.
ultimate_cron.job.node_queue_worker.yml
Need to place inside install folder. If the module is already enabled, we need to import this file using configuration management interface.
Now create a class to call processItem() that will process the queue.
ArticleQueueProcessor.php
In this method, we are going to define the processItem() method to create a node from the data received in API.
Once cron run is completed, an article will be created on the site.
To check this manually, we can navigate to “admin/config/system/cron/jobs” and run the cron “Article Queue Worker”. After cron run navigate to “admin/config/system/queue-ui” and select “Task Worker: Node” and run batch process. Once the process is over your content will be created on the site.
That’s it!
See how easily you can deploy Queue Worker/API in Drupal to handle a number of resource intensive tasks at a later point of time. With Queue Worker, items can be added by passing an arbitrary data object. Also, using this API, you can easily manage your resource intensive tasks on your website.
Hope you can now implement Queue Worker/API in Drupal. If you have any suggestions or queries please comment down let me try to answer.
