Contents
Introduction
This tutorial explains how to build a simple, reliable task queue in WordPress using transients to store queued tasks and WP_CRON to process them in the background. Youll get a complete, ready-to-use implementation with enqueueing, batched processing, locking to avoid race conditions, retry/backoff on failures, activation/deactivation hooks, and suggestions for production hardening.
Why use transients WP_CRON
- Transients are a simple way to store data in the WordPress database (or in an object cache) with optional expiration. They are convenient for keeping a small-to-medium sized queue without adding custom DB tables.
- WP_CRON provides periodic background execution on WordPress sites without system cron access, allowing the queue processor to run at regular intervals.
- This combination is simple, requires no new infrastructure, and fits many use-cases where tasks are not ultra-latency-sensitive and throughput is modest.
Design overview
- Tasks are stored as an array inside a single transient keyed for your plugin (or multiple chunked transients if you expect very large queues).
- Each task is a structured array: id, hook/callback, args, attempts, max_attempts, backoff seconds, available_at timestamp, created timestamp, optional meta/state.
- WP_CRON schedules a worker to run frequently (every minute or every few minutes) and process up to a configurable batch size per run.
- A locking mechanism prevents concurrent workers from processing the same queue on racey setups (multiple web servers or overlapping CRON runs).
- On failure the task gets re-queued with an incremented attempts count and a delay (backoff). When attempts exceed max, the task is moved to a fail state or dropped.
Implementation
Below is a full, practical implementation you can copy into a plugin file. It includes:
- Queue constants and helpers
- Enqueue API
- Worker and batch processing
- Locking using DB add_option / delete_option
- Activation / deactivation hooks to schedule and clear CRON events
- Example task handler
Place the code in your plugin file. All code examples are between the required
...
tags.
task_id,
hook => (string) hook,
args => (array) args,
attempts => 0,
max_attempts=> isset( opts[max_attempts] ) ? (int) opts[max_attempts] : 3,
backoff => isset( opts[backoff] ) ? (int) opts[backoff] : 60, // seconds
available_at=> isset( opts[available_at] ) ? (int) opts[available_at] : time(),
created_at => time(),
)
// Add to end of queue
queue[] = task
// Save transient. No expiration (0) means keep until deleted.
set_transient( MYQUEUE_TRANSIENT_KEY, queue, 0 )
// Ensure cron is scheduled (safety - optional)
if ( ! wp_next_scheduled( MYQUEUE_CRON_HOOK ) ) {
myqueue_schedule_cron()
}
return task_id
}
/
Acquire a simple DB-backed lock using add_option (atomic).
Returns true if locked successfully, false if the lock exists.
/
function myqueue_acquire_lock() {
// Use add_option - it will fail if the option already exists.
created = add_option( MYQUEUE_LOCK_OPTION, time(), , no )
if ( false === created ) {
// Already exists
// Option may be stale check TTL and force delete if old
lock_time = (int) get_option( MYQUEUE_LOCK_OPTION, 0 )
if ( lock_time ( time() - lock_time ) > MYQUEUE_LOCK_TTL ) {
// Stale lock - remove and attempt to add again
delete_option( MYQUEUE_LOCK_OPTION )
created = add_option( MYQUEUE_LOCK_OPTION, time(), , no )
return ( false !== created )
}
return false
}
return true
}
/
Release the lock.
/
function myqueue_release_lock() {
delete_option( MYQUEUE_LOCK_OPTION )
}
/
The WP_CRON hook handler - will attempt to process a batch.
/
function myqueue_process_task_queue_cron() {
// Try to acquire lock - prevents overlapping processing
if ( ! myqueue_acquire_lock() ) {
return // someone else is processing
}
try {
queue = get_transient( MYQUEUE_TRANSIENT_KEY )
if ( ! is_array( queue ) empty( queue ) ) {
// nothing to do
delete_transient( MYQUEUE_TRANSIENT_KEY )
myqueue_release_lock()
return
}
now = time()
new_queue = array()
processed = 0
foreach ( queue as task ) {
// If weve reached batch size, keep remaining tasks for next run
if ( processed >= MYQUEUE_BATCH_SIZE ) {
new_queue[] = task
continue
}
// Skip tasks scheduled for future (available_at)
if ( isset( task[available_at] ) task[available_at] > now ) {
new_queue[] = task
continue
}
// Attempt to run task
success = false
try {
// We trigger an action with hook name. Consumers can add_action( myplugin_task_foo, handler )
hook_name = myplugin_task_ . task[hook]
/
Handlers are expected to:
- return true on success or false on failure
- OR throw an Exception on fatal error (we catch below)
We will gather the return value using do_action_ref_array is not designed to return values.
Instead, encourage handlers to call myqueue_task_result( task_id, result )
For simplicity here we call apply_filters to allow a single callable returning boolean.
/
// If handler wants to be a callable string you can use apply_filters:
handler_result = apply_filters( hook_name, null, task[args], task )
// Handler should return true on success, false or null for failure.
if ( true === handler_result ) {
success = true
} else {
// If no filter returned true, attempt to call a globally available function with the same name as hook.
if ( is_callable( task[hook] ) ) {
ret = call_user_func_array( task[hook], (array) task[args] )
if ( ret === true ) {
success = true
}
} else {
// allow actions: action handlers may not return. Trigger an action and assume success.
do_action_ref_array( hook_name, array( task[args], task ) )
// For action-based processing we default to success (handler should manage persistence / errors).
success = true
}
}
} catch ( Exception e ) {
// treat as failure
success = false
// Optionally log the exception
if ( function_exists( error_log ) ) {
error_log( myqueue task exception: . e->getMessage() )
}
}
if ( success ) {
// Task done - do not requeue
processed
continue
}
// Failure handling - retry with backoff or drop
task[attempts] = isset( task[attempts] ) ? (int) task[attempts] 1 : 1
if ( isset( task[max_attempts] ) task[attempts] >= task[max_attempts] ) {
// Move to dead-letter area or drop here well send an action to allow logging
do_action( myplugin_task_failed, task )
// Drop the task (do not re-add to queue)
continue
} else {
// Exponential backoff: backoff attempts
backoff = isset( task[backoff] ) ? (int) task[backoff] : 60
task[available_at] = time() ( backoff task[attempts] )
new_queue[] = task
processed
continue
}
} // foreach
// Save new queue
if ( empty( new_queue ) ) {
delete_transient( MYQUEUE_TRANSIENT_KEY )
} else {
set_transient( MYQUEUE_TRANSIENT_KEY, new_queue, 0 )
}
} finally {
// Always release lock even on fatal errors
myqueue_release_lock()
}
}
/
Schedule CRON event for processing.
/
function myqueue_schedule_cron() {
if ( ! wp_next_scheduled( MYQUEUE_CRON_HOOK ) ) {
// Ensure schedule exists - here we schedule every minute (add custom schedule below)
wp_schedule_event( time(), every_minute, MYQUEUE_CRON_HOOK )
}
}
/
Unschedule CRON events for cleanup.
/
function myqueue_unschedule_cron() {
timestamp = wp_next_scheduled( MYQUEUE_CRON_HOOK )
if ( timestamp ) {
wp_unschedule_event( timestamp, MYQUEUE_CRON_HOOK )
}
}
/
Add custom cron schedule: every_minute
/
add_filter( cron_schedules, myqueue_cron_schedules )
function myqueue_cron_schedules( schedules ) {
if ( ! isset( schedules[every_minute] ) ) {
schedules[every_minute] = array(
interval => 60,
display => __( Every Minute ),
)
}
return schedules
}
// Hook our cron handler
add_action( MYQUEUE_CRON_HOOK, myqueue_process_task_queue_cron )
// Activation / deactivation hooks
register_activation_hook( __FILE__, myqueue_activate )
register_deactivation_hook( __FILE__, myqueue_deactivate )
function myqueue_activate() {
myqueue_schedule_cron()
}
function myqueue_deactivate() {
myqueue_unschedule_cron()
// Optional: remove transient and lock
delete_transient( MYQUEUE_TRANSIENT_KEY )
delete_option( MYQUEUE_LOCK_OPTION )
}
/
Example usage:
Enqueue a task:
myqueue_enqueue_task( send_welcome_email, array( user_id => 123 ), array( max_attempts => 5, backoff => 120 ) )
Handle the task by adding a filter that returns true on success:
add_filter( myplugin_task_send_welcome_email, my_send_welcome_email_handler, 10, 3 )
function my_send_welcome_email_handler( return, args, task ) {
user_id = isset( args[user_id] ) ? intval( args[user_id] ) : 0
if ( ! user_id ) {
return false
}
// run email sending return true if OK, false on failure
// wp_mail( ... )
return true
}
Or use action handlers:
add_action( myplugin_task_send_welcome_email, my_send_welcome_email_action_handler, 10, 2 )
function my_send_welcome_email_action_handler( args, task ) {
// Do not return anything the queue will assume success.
}
/
?>
Explanation of key parts
- Task structure: id, hook, args, attempts, max_attempts, backoff, available_at, created_at. This allows retrying and delaying failed tasks.
- Storage: The whole queue is a single transient. set_transient(…, 0) stores it without expiration. For very large queues you should chunk into multiple transients.
- Locking: We use add_option/delete_option for the lock. add_option returns false if an option already exists — this is an atomic DB operation that works reliably on standard WP installs and prevents two processes from processing the queue at the same time. We also detect stale locks and recover.
- Handlers: The worker tries two strategies: a filter that returns true on success, or action handlers. Filters allow handlers to explicitly signal success actions are treated as implicitly successful (use actions only when handlers manage errors themselves).
- Backoff retries: Failed tasks are retried with exponential backoff (backoff attempts). When attempts exceed max_attempts a myplugin_task_failed action is fired so you can log or alert.
Enqueuing tasks
Call myqueue_enqueue_task( hook, args, opts ) from anywhere (plugin code, REST endpoint, admin UI) to add a task. Do not store closures/anonymous functions in task args since serialization will fail pass simple arrays, scalars, IDs or strings referencing functions/hooks.
Adding handlers
You can add either:
- Filter-based handler that returns true or false: add_filter( myplugin_task_foo, handler, 10, 3 ) and return true on success.
- Action-based handler: add_action( myplugin_task_foo, handler, 10, 2 ). Actions do not return values — worker treats them as successful unless exceptions occur. Prefer filters when you need explicit success detection.
Advanced topics and production hardening
1) When a single transient is not enough
If you expect large volumes of tasks (hundreds or thousands) storing all tasks in a single transient can become a bottleneck (large option size, serialization costs, risk of race conditions). Alternatives:
- Chunk the queue into multiple transients (e.g., myqueue_chunk_1, myqueue_chunk_2) and cycle through them.
- Create a custom DB table for tasks (recommended for medium/large queues) with indexed status and scheduled time.
- Use a persistent external queue (Redis, Amazon SQS, RabbitMQ) for high throughput.
2) Locking strategies
Our add_option approach is DB-backed and works even without persistent object cache. If you have a persistent object cache (memcached/redis) you can use wp_cache_add as an atomic operation — faster but only works when persistent cache is enabled. Ensure you have lock TTLs and stale lock recovery.
3) Multiple workers horizontal scaling
Multiple web nodes polling WP_CRON can overlap. Use strong locking (DB locks or external distributed locks) and consider reducing cron frequency or using a single master worker (via a remote worker or system cron with WP-CLI).
4) Batch size and execution time
Choose a batch size based on the expected runtime of a task and PHP max_execution_time. If tasks are heavy (long-running) process fewer per batch. If many short tasks, increase batch size. WP_CRON runs inside web requests and must finish before request timeouts.
5) Error handling and visibility
- Implement myplugin_task_failed action to log failures into a dedicated log or DB table for operator review.
- Consider adding a dead-letter queue transient or option to store permanently failed tasks with metadata.
6) Security
- If you expose enqueueing endpoints (REST or admin AJAX), require proper capabilities and nonces to avoid untrusted users flooding your queue.
- Sanitize and validate task arguments. Avoid storing user-supplied executable strings or closures.
7) Testing and debugging
- Temporarily reduce cron interval to test quickly. You can trigger the worker manually by calling myqueue_process_task_queue_cron() from an admin action, but be careful with locks.
- Log important events (enqueue, process start, task success, failure) with error_log or a logger.
- Use WP-CLI to trigger processing: wp eval myqueue_process_task_queue_cron() — this is useful for manual runs or system cron integration.
8) WP-CLI integration example
You can add a WP-CLI command to process the queue. Doing so lets you run the queue from system cron reliably (preferable for high-volume sites).
Performance and limits
- Transients stored in options table become large options. Avoid extremely large stored arrays or frequent writes that cause heavy DB I/O.
- Object caches (Memcached, Redis) vastly improve transient performance. If you have them, use wp_cache_add for locks.
- WP_CRON is triggered by visits. On low-traffic sites cron runs may be delayed consider system cron WP-CLI for reliable scheduling.
Other patterns and alternatives
- Use a dedicated DB table if you need complex queries, status transitions, or high throughput.
- Use external queue services (SQS, Redis queues) for distributed, scalable systems.
- For short-lived inline tasks (e.g., sending one email), consider immediate processing to keep the queue simple — but avoid long-running work in a request.
Common pitfalls
- Storing closures/objects in tasks — they wont serialize correctly.
- Not handling stale locks, leading to deadlocks where no worker runs.
- Too-large transients causing heavy DB/serialization overhead.
- Assuming WP_CRON runs exactly on schedule on low-traffic sites.
Summary
Using transients and WP_CRON you can implement a simple, effective task queue for WordPress suitable for low-to-moderate load. The example above gives a complete, practical foundation: enqueue tasks, scheduled processing, safe locking, batched execution, retry with backoff, and hooks for failure handling. For production-level queues that require high throughput, strict latency, or complex querying, consider migrating to a dedicated table or external queue service and prefer system cron or WP-CLI for reliable scheduling.
Useful links
|
|
Acepto donaciones de BAT's mediante el navegador Brave 🙂 |