What is Celery?

Celery is an asynchronous task queue that allows Django applications to handle long-running tasks in the background without blocking the main application. It’s commonly used for sending emails, processing large datasets, handling scheduled tasks, and more.

How Celery Works in Django

Celery follows a Producer-Consumer architecture. The Django application (Producer) adds tasks to a message broker (like Redis), and Celery Workers (Consumers) execute those tasks in the background.

Celery Architecture

1. Producer (Django App): Sends tasks to the broker.
2. Broker (Redis/RabbitMQ): Stores tasks temporarily.
3. Worker (Celery Worker Process): Picks up and executes tasks from the broker.
4. Result Backend (Redis/Database): Stores task results and status.

Step-by-Step Celery Implementation in Django

Install Celery and Redis

pip install celery redis        

Configure Celery in Django

# settings.py
CELERY_BROKER_URL = 'redis://localhost:6379/0'  # Redis as a broker
CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'        

Create Celery Instance

# project/celery.py
import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings')
app = Celery('project')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()

-------------------------------------------
# Modify (project/__init__.py)

 from .celery import app as celery_app
__all__ = ['celery_app']        

Define a Celery Task

# reports/tasks.py
from celery import shared_task
import time

@shared_task
def generate_report(report_type):
    time.sleep(10)  # Simulating a time-consuming task
    return f'Report {report_type} generated successfully!'        

Trigger Celery Task in Django View

# reports/views.py
from django.http import JsonResponse
from .tasks import generate_report

def request_report(request):
    task = generate_report.delay('sales')
    return JsonResponse({'message': 'Report is processing...', 'task_id': task.id})        

Start Redis & Celery Workers

# Start Redis server:
redis-server

# Start Celery Worker
celery -A project worker --loglevel=info        

Real-World Use Cases for Celery

?? Sending Emails: Send bulk emails without blocking user requests.

?? Generating Reports: Process large CSV/Excel files asynchronously.

?? Data Processing: Run background jobs like database cleanup.

?? Image Processing: Resize images or generate thumbnails.

?? Periodic Tasks: Schedule daily tasks like sending reminders.

Celery is an essential tool for Django developers who need to handle background tasks efficiently. By integrating Celery with Django, you can improve performance, scale tasks, and automate workflows.