Introducing Vapor Queues

In the world of web development, maintaining a responsive user experience is crucial. As applications grow in complexity, certain tasks—like sending emails or processing large datasets—can slow down the main request thread. This is where Vapor Queues come in, providing a robust solution for offloading tasks to background workers. In this article, we’ll explore the key features of Vapor Queues and how to implement them using Swift’s async/await syntax.

What Are Vapor Queues?

Vapor Queues is a pure Swift queuing system designed to handle tasks outside the main request thread. This package is beneficial for:

• Sending emails asynchronously
• Performing long-running database operations
• Ensuring job integrity and resilience
• Improving response times by handling non-critical tasks in the background
• Scheduling jobs for future execution

Vapor Queues shares similarities with Ruby's Sidekiq, offering features such as:

• Safe handling of shutdown signals (SIGTERM/SIGINT)
• Different queue priorities for various tasks
• Reliable job processing to manage unexpected failures
• A maxRetryCount feature to retry failed jobs
• Utilization of NIO for performance across multiple cores
• Support for scheduling recurring tasks

Supported Drivers

Vapor Queues officially supports the Redis driver through the QueuesRedisDriver package. Community-driven drivers are also available, including:

• QueuesMongoDriver
• QueuesFluentDriver

Getting Started with Vapor Queues

To start using Vapor Queues, follow these steps:

Step 1: Add the Redis Driver to Your Project

In your Swift package manifest file, add the Redis driver as a dependency:

// swift-tools-version:5.8 
import PackageDescription 
let package = Package( 
    name: "MyApp", 
    dependencies: [ 
        .package(url: "https://github.com/vapor/queues-redis-driver.git", from: "1.0.0"),
        ], 
    targets: [ 
        .executableTarget(
            name: "App", dependencies: [ 
                .product(name: "QueuesRedisDriver", package: "queues-redis-driver") 
            ]), 
        .testTarget(
            name: "AppTests", 
            dependencies: [
                .target(name: "App")
            ]), 
        ] 
)        

After saving the changes, Xcode will automatically fetch the new dependency. Alternatively, run swift package resolve in the terminal.

Step 2: Configure Queues

Next, configure the queues in your configure.swift file:

import QueuesRedisDriver 
try app.queues.use(.redis(url: "redis://127.0.0.1:6379"))        

Step 3: Register a Job

Define a job and add it to your application:

struct Email: Codable { 
    let to: String 
    let message: String 
} 

struct EmailJob: AsyncJob { 
    typealias Payload = Email 
    func dequeue(_ context: QueueContext, _ payload: Email) async throws { 
        // Code to send the email 
    } 
    func error(_ context: QueueContext, _ error: Error, _ payload: Email) async throws { 
        // Handle errors if necessary 
    } 
} 
// Register the job 
let emailJob = EmailJob() app.queues.add(emailJob)        

Running Workers

To process queued jobs, you need to run a worker. Use the command:

swift run App queues        

For specific queue types, you can run:

swift run App queues --queue emails        

In production, ensure workers remain active. For example, on Heroku, specify worker dynos in your Profile.

Dispatching Jobs

You can dispatch jobs from route handlers. Here’s how to do it using async/await:

app.get("email") { req async throws -> String in 
    try await req.queue.dispatch( 
        EmailJob.self, 
        .init(to: "[email protected]", message: "Hello!") 
    ) 
return "Email dispatched" 
}        

You can also set a maximum retry count or specify a delay before the job runs:

app.get("email") { req async throws -> String in 
    let futureDate = Date(timeIntervalSinceNow: 60 * 60 * 24) // 1 day 
    try await req.queue.dispatch( 
        EmailJob.self, 
        .init(to: "[email protected]", message: "Hello!"), 
        maxRetryCount: 3, delayUntil: futureDate 
    ) 
return "Email scheduled for dispatch" 
}        

Prioritizing Jobs Continued

To route a job to a specific queue, you first need to extend QueueName to define your custom queues. Once done, you can dispatch the job to the desired queue:

// Dispatching the email job to the 'emails' queue 
app.get("email") { req async throws -> String in 
    let futureDate = Date(timeIntervalSinceNow: 60 * 60 * 24) // Schedule for 1 day later
    try await req .queues(.emails) // Specify the 'emails' queue 
        .dispatch( 
        EmailJob.self, 
        .init(to: "[email protected]", message: "Hello!"), 
        maxRetryCount: 3, 
        delayUntil: futureDate 
    ) 
    return "Email job dispatched to the emails queue" 
}        

Working with Application Context

If you need to dispatch jobs from a command context (where you don't have access to the request object), you can still utilize the queues through the application instance:

struct SendEmailCommand: AsyncCommand { 
    func run(using context: CommandContext, signature: Signature) async throws { 
    let futureDate = Date(timeIntervalSinceNow: 60 * 60 * 24) // Schedule for 1 day 
    try await context 
        .application 
        .queues 
        .queue(.emails) // Access the 'emails' queue 
        .dispatch( 
            EmailJob.self, 
            .init(to: "[email protected]", message: "Hello!"), 
            maxRetryCount: 3, 
            delayUntil: futureDate 
            ) 
    } 
}        

Conclusion

Vapor Queues provide a powerful way to manage background tasks in Swift applications, enhancing performance and user experience. With the ability to handle job retries, manage different job priorities, and utilize Swift's async/await syntax, developers can easily integrate asynchronous processes into their applications.

By following this guide, you've seen how to set up Vapor Queues, register jobs, dispatch tasks, and prioritize work based on application needs. Whether you're sending emails, processing data, or scheduling tasks, Vapor Queues is a robust addition to your development toolbox.

As you explore more complex workflows and architectures, consider leveraging the community drivers for MongoDB or Fluent to fit your project requirements. Happy coding!

#Swift

#iOS

#Vapor

#Backend