ask · February 15, 2012 10:43 · Feb 15, 2012 · Feb 14, 2012
diff --git a/gistfile1.txt b/gistfile1.txt
@@ -1,11 +1,91 @@
-When I'm configuring in a Django app, what is the purpose of the djcelery models? Right now tables get created, but nothing flows into them. Is this the database backend replacement for Redis and RabbitMQ? Or is it something else?
+-When I'm configuring in a Django app, what is the purpose of the djcelery models? Right --now tables get created, but nothing flows into them. Is this the database backend
+---- replacement for Redis and RabbitMQ? Or is it something else?
 
-Why do workers delete items from the queue if they are unable to process them (i.e. task wasn't found)?
+- Several database tables are used:
 
-What is the best way to identify if a task, based on id, exists in the queue and what its status is?
+  * Monitoring
 
-How do I know if a task id is even valid?
+    When you use the django-admin monitor the cluster state is written
+    to the TaskState and WorkerState tables.
 
-What is the preferred way to run 2 celeryd processes that are intended to handle different tasks? (i.e. proc 1 doesn't handle tasks that proc 2 is responsible for and vice versa)
+  * Periodic tasks
 
-These are just a few questions I've had to start. Don't get me wrong, I got it to work and it's really nice in that regard. I just can't find information on the above issues. Thanks for reaching out :-)
+    When the database-backed schedule is used this takes the periodic task
+    schedule from the PeriodicTask model, there are also several other
+    helper tables (IntervalSchedule, CrontabSchedule, PeriodicTasks).
+
+  * Task results
+
+    The database result backend is enabled by default when using django-celery
+    (this is for historical reasons, and thus for backward compatibility).
+
+    The results are stored in the TaskMeta and TaskSetMeta models.
+    *these tables are not created if another result backend is configured*.
+
+Django apps usually do create all tables, even if they are not going to be used,
+in fact it is uncommon to do like django-celery and only create the result tables
+if configured to use the database result backend.
+
+-Why do workers delete items from the queue if they are unable to process them (i.e. task -wasn't found)?
+
+The worker discards unknown tasks, messages with encoding errors and messages that doesn't
+contain the proper fields (as per the task message protocol).
+
+If it did not ack (delete) them they would be redelivered again and again causing a loop.
+
+There has been talk about moving these messages to a dead-letter queue, but that has
+not yet been implemented.
+
+-What is the best way to identify if a task, based on id, exists in the queue and what its -status is?
+
+Use task states (results): http://docs.celeryproject.org/en/latest/userguide/tasks.html#task-states
+
+- How do I know if a task id is even valid?
+
+If what you mean to ask is, "how do I know that a task with a particular id exists?",
+then the answer is that you can't.
+
+Every unknown task id in Celery is assumed to be PENDING, this is an optimization
+so that it doesn't need to write a state every time it sends a task message.
+But the main point is that asking this question is almost always an anti-pattern:
+whenever you think that you need to know if a task exists or not, you are solving
+a problem the wrong way. See http://docs.celeryproject.org/en/latest/userguide/tasks.html#state
+
+-What is the preferred way to run 2 celeryd processes that are intended to handle
+-different tasks? (i.e. proc 1 doesn't handle tasks that proc 2 is responsible for and vice versa)
+
+Tasks are sent to queues, so you can sort the tasks into topics, let's say "hipri" and "lopri".
+
+To tell a worker to consume from a particular queue you can use the "-Q" argument:
+
+    celeryd -Q hipri
+    celeryd -Q hipri,lopri
+
+To properly route the tasks you should use the CELERY_ROUTES setting:
+
+    CELERY_ROUTES = {"tasks.important": {"queue": "hipri"},
+                     "tasks.insignificant": {"queue": "lowpri"}}
+
+See the routing user guide for more information:
+http://docs.celeryproject.org/en/latest/userguide/routing.html
+
+TIP: celeryd-multi can be used to start multiple celeryd instances:
+
+    $ celeryd-multi start w1 w2  -Q:w1 hipri -Q:w2 hipri,lopri
+    $ manage.py celeryd_multi start w1 w2 -Q:w1 hipri -Q:w2 hipri,lopri
+
+or when using the generic init script, the following can be used in your `/etc/default/celeryd`:
+
+    CELERYD_NODES="w1 w2"
+    CELERYD_OPTS="-Q:w1 hipri -Q:w2 hipri,lopri
+
+(celeryd-multi examples: http://docs.celeryproject.org/en/latest/reference/celery.bin.celeryd_multi.html)
+
+-These are just a few questions I've had to start. Don't get me wrong, I got it to work ---and it's really nice in that regard. I just can't find information on the above issues. --Thanks for reaching out :-)
+
+It is hard for people involved in a project to write good documentation for it,
+as with so much knowledge about the topic it is almost impossible to know what
+new users need to know.
+
+Thank you!
+I will find a way to add this to the documentation.
diff --git a/gistfile1.txt b/gistfile1.txt
@@ -0,0 +1,11 @@
+When I'm configuring in a Django app, what is the purpose of the djcelery models? Right now tables get created, but nothing flows into them. Is this the database backend replacement for Redis and RabbitMQ? Or is it something else?
+
+Why do workers delete items from the queue if they are unable to process them (i.e. task wasn't found)?
+
+What is the best way to identify if a task, based on id, exists in the queue and what its status is?
+
+How do I know if a task id is even valid?
+
+What is the preferred way to run 2 celeryd processes that are intended to handle different tasks? (i.e. proc 1 doesn't handle tasks that proc 2 is responsible for and vice versa)
+
+These are just a few questions I've had to start. Don't get me wrong, I got it to work and it's really nice in that regard. I just can't find information on the above issues. Thanks for reaching out :-)