No free threads available for launching jobs

Error Type: 
Contributed Module
Module: 
Severity: 
Warning
Origin: 
watchdog
File name: 
serial.class.php
Description: 

The Ultimate Cron module uses the concept of threads to control the amount of resource that should be devoted to processing queued tasks at any one time.

When this message appears in your watchdog reports it means that a cron cycle has been initiated but the maximum number of threads are already running meaning that no new threads could be created to process queued items on this cycle.

Cause: 

It can be perfectly normal to see this entry in your watchdog log. For example, if you have large batch operation that runs at the end of each month several thousand items might get queued for processing. At the same time lets assume that you have configured cron to be invoked every minute, however it takes more than one minute to complete processing on a chunk of the queue. This would result is a new thread being used before the previously started thread has been completed. Or in other words we now have multiple threads running concurrently.

The warning message tells us that cron resources are being throttled in order to protect normal site performance. The result is that cron will take longer to clear the queued items, but it will clear them eventually, and people using the site will not have experienced a degraded service.

Solution: 

If your server is operating well within its capacity, with unused RAM and CPU then there is a very simple solution - allow more threads to be used.

The Ultimate Cron module allows the site administrator to configure the maximum number of threads that can be active at any one time. To do this navigate as follows: admin/config/system/cron/settings/queue

From this page you can increase the value in the "Threads" field and then save your change. We suggest increasing this value by one initially, the rationale being that you want to use the fewest number of threads possible in order to leave as much server resource free for handling page requests as possible.

However, this problem can be indicative of a problem appearing on the horizon and increasing the maximum allowed threads without considering other possibilities could exacerbate problems in your Drupal install. One should stay alert to the possibility of any of the following:

Hardware Constraints
If your server is already nearing its limits in terms of RAM and CPU usage then increasing the number of cron threads is likely to mean that your server has very little resource left to service normal page requests when these long cron operations are in progress. The consequence of this could be that users of your site experience very sluggish performance and possibly error screens.

Software Problem
The reason that multiple threads are being used is that you have batch operations that are taking some time to complete. This would indicate that you are trying to process a large amount of data that might involve lots of database access of external API calls. This can be completely normal, but if you are not aware of any processing that you would expect to complete in more than a few seconds it could be that some buggy code in a module is processing data very inefficiently. In this situation the software should be fixed first to avoid problems that might prevent you from scaling up from your service later.

Something else chewing up system resources
Maybe the cron initiated batch processes are not the problem at all and in fact some other problem with your Drupal install is meaning that overall performance is very poor, page requests take a long time to load and the server is running at maximum capacity. In this instance the warning from the Ultimate Cron module might be a problem that is secondary to some other issue that must be fixed. Good candidates to consider are:

  • Pages that host a View which display a large number of entities
  • Database locks due to many entities and their field being written to the database at the same time

 

 

Expedite

Our aim is to add enough detailed information to each of the Drupal error messages that Drupal developers will be able to fix their own problems. This exercise will take some time to complete, at the last count we had almost 700 error messages in the system.

If your site is suffering from this error message please contact us and ask us to expedite the solution to this error and we will do our best to add a solution as soon as possible.