Sending messages using a background task

The operation of sending an email message can take up to several seconds because of the need to log into and communicate with a potentially remote SMTP server. To keep the user from having to wait for the send operation to complete, it is sometimes desirable to queue the email to be sent at a later time via a background task. As described in Chapter 4, this can be done by setting up a homemade task queue or using the web2py scheduler. Here we provide an example using a homemade task queue.

First, in a model file within our application, we set up a database model to store our email queue:

  1. db.define_table('queue',
  2.     Field('status'),
  3.     Field('email'),
  4.     Field('subject'),
  5.     Field('message'))

From a controller, we can then enqueue messages to be sent by:

  1. db.queue.insert(status='pending',
  2.                 email='you@example.com',
  3.                 subject='test',
  4.                 message='test')

Next, we need a background processing script that reads the queue and sends the emails:

  1. ## in file /app/private/mail_queue.py
  2. import time
  3. while True:
  4.     rows = db(db.queue.status=='pending').select()
  5.     for row in rows:
  6.         if mail.send(to=row.email,
  7.             subject=row.subject,
  8.             message=row.message):
  9.             row.update_record(status='sent')
  10.         else:
  11.             row.update_record(status='failed')
  12.         db.commit()
  13.     time.sleep(60) # check every minute

Finally, as described in Chapter 4, we need to run the mail_queue.py script as if it were inside a controller in our app:

  1. python web2py.py -S app -M -R applications/app/private/mail_queue.py

where -S app tells web2py to run “mail_queue.py” as “app”, -M tells web2py to execute models.

Here we assume that the mail object referenced in “mail_queue.py” is defined in a model file in our app and is therefore available in the “mail_queue.py” script because of the -M option. Also notice that it is important to commit any change as soon as possible in order not to lock the database to other concurrent processes.

As noted in Chapter 4, this type of background process should not be executed via cron (except perhaps for cron @reboot) because you need to be sure that no more than one instance is running at the same time.

Note, one drawback to sending email via a background process is that it makes it difficult to provide feedback to the user in case the email fails. If email is sent directly from the controller action, you can catch any errors and immediately return an error message to the user. With a background process, however, the email is sent asynchronously, after the controller action has already returned its response, so it becomes more complex to notify the user of a failure.