celery-beat - Database-backed Periodic Tasks — django_celery_beat 2.5.0 documentation (original) (raw)

Version:

2.5.0

Web:

http://django-celery-beat.readthedocs.io/

Download:

http://pypi.python.org/pypi/django-celery-beat

Source:

http://github.com/celery/django-celery-beat

Keywords:

django, celery, beat, periodic task, cron, scheduling

About

This extension enables you to store the periodic task schedule in the database.

The periodic tasks can be managed from the Django Admin interface, where you can create, edit and delete periodic tasks and how often they should run.

Using the Extension

Usage and installation instructions for this extension are available from the Celery documentation.

Important Warning about Time Zones

Warning

If you change the Django TIME_ZONE setting your periodic task schedule will still be based on the old timezone.

To fix that you would have to reset the “last run time” for each periodic task:

from django_celery_beat.models import PeriodicTask, PeriodicTasks PeriodicTask.objects.update(last_run_at=None) PeriodicTasks.changed()

Note that this will reset the state as if the periodic tasks have never run before.

Models

Whenever you update a PeriodicTask, a counter in this table is also incremented, which tells the celery beat service to reload the schedule from the database.

If you update periodic tasks in bulk, you will need to update the counter manually:

from django_celery_beat.models import PeriodicTasks PeriodicTasks.changed()

Example creating interval-based periodic task

To create a periodic task executing at an interval you must first create the interval object:

from django_celery_beat.models import PeriodicTask, IntervalSchedule

executes every 10 seconds.

schedule, created = IntervalSchedule.objects.get_or_create( ... every=10, ... period=IntervalSchedule.SECONDS, ... )

That’s all the fields you need: a period type and the frequency.

You can choose between a specific set of periods:

Note

If you have multiple periodic tasks executing every 10 seconds, then they should all point to the same schedule object.

There’s also a “choices tuple” available should you need to present this to the user:

IntervalSchedule.PERIOD_CHOICES

Now that we have defined the schedule object, we can create the periodic task entry:

PeriodicTask.objects.create( ... interval=schedule, # we created this above. ... name='Importing contacts', # simply describes this periodic task. ... task='proj.tasks.import_contacts', # name of task. ... )

Note that this is a very basic example, you can also specify the arguments and keyword arguments used to execute the task, the queue to send it to [1], and set an expiry time.

Here’s an example specifying the arguments, note how JSON serialization is required:

import json from datetime import datetime, timedelta

PeriodicTask.objects.create( ... interval=schedule, # we created this above. ... name='Importing contacts', # simply describes this periodic task. ... task='proj.tasks.import_contacts', # name of task. ... args=json.dumps(['arg1', 'arg2']), ... kwargs=json.dumps({ ... 'be_careful': True, ... }), ... expires=datetime.utcnow() + timedelta(seconds=30) ... )

Example creating crontab-based periodic task

A crontab schedule has the fields: minute, hour, day_of_week,day_of_month and month_of_year, so if you want the equivalent of a 30 * * * * (execute at 30 minutes past the hour every hour) crontab entry you specify:

from django_celery_beat.models import CrontabSchedule, PeriodicTask schedule, _ = CrontabSchedule.objects.get_or_create( ... minute='30', ... hour='', ... day_of_week='', ... day_of_month='', ... month_of_year='', ... )

Then to create a periodic task using this schedule, use the same approach as the interval-based periodic task earlier in this document, but instead of interval=schedule, specify crontab=schedule:

PeriodicTask.objects.create( ... crontab=schedule, ... name='Importing contacts', ... task='proj.tasks.import_contacts', ... )

Temporarily disable a periodic task

You can use the enabled flag to temporarily disable a periodic task:

periodic_task.enabled = False periodic_task.save()

Example running periodic tasks

The periodic tasks still need ‘workers’ to execute them. So make sure the default Celery package is installed. (If not installed, please follow the installation instructions here: GitHub project celery/celery)

Both the worker and beat services need to be running at the same time.

  1. Start a Celery worker service (specify your Django project name):
    $ celery -A [project-name] worker --loglevel=info
  2. As a separate process, start the beat service (specify the Django scheduler):
    $ celery -A [project-name] beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler

OR you can use the -S (scheduler flag), for more options see celery beat --help):

$ celery -A [project-name] beat -l info -S django

OR you can set the scheduler through Django’s settings:

CELERY_BEAT_SCHEDULER = 'django_celery_beat.schedulers:DatabaseScheduler'

Also, as an alternative, you can run the two steps above (worker and beat services) with only one command (recommended for development environment only):

$ celery -A [project-name] worker --beat --scheduler django --loglevel=info

  1. Now you can add and manage your periodic tasks from the Django Admin interface.

Working with django-celery-results

Now you can store PeriodicTask.nameto django-celery-results (TaskResult.periodic_task_name).

Suppose we have two periodic tasks, their schedules are different, but the tasks are the same.

name task args schedule
schedule1 schedule2 some.celery.task some.celery.task (1,) (2,) every hour every 2 hours

Now you can distinguish the source of the task from the results by the periodic_task_name field.

id task_name periodic_task_name
uuid1 uuid2 uuid3 uuid4 some.celery.task some.celery.task some.celery.task some.celery.task schedule1 schedule1 schedule2 schedule2

(more technical details here: GitHub PR #477, GitHub PR #261)

Contents

Indices and tables