leela email service

leela is an email sending service that allows you to create and customize your HTML emails easily, without coding. It allows embedded images and attachments.

How leela works

It is built on:

• Python 3.4
• Django
• Postgres
• Mandrill
• RabbitMQ

leela has three main features:

• Admin site, that you'll use to manage your email configurations, and check out the generated emails. You will find it at /admin/ .
• Queue consumer, that will consume messages from a queue system to send configured emails.
• REST API, to make queries about sent emails.

The backend is mainly comprised of two types:

• EmailKind represents a configured type of email. These are created manually through the admin site.
• EmailEntry represents an actual email. It is generated by, and only by, the queue consumer. It is related to one EmailKind. Informally speaking, you could say that an EmailEntry is an instance of an EmailKind.

How to send your email

With leela there's no need to write any code to be able to send full featured HTML emails. The steps to configure a new email are:

1. Access the admin website

Go to /admin/ => "Email kinds" => "Add email kind". There it is the form to create a new EmailKind. An EmailKind is identified by both its name and language (both together must be unique). You can change the allowed languages in the project settings.

2. Create your EmailKind

Fill up the form with the desired templates. Plain template is mandatory, given that many email clients do not support HTML templates. The template and JSON fields use the CodeMirror editor to make it easier to write code there. The templates are rendered against the context defined in the parameters, check below.

In order to embed images into the Template field you have to use the ContentID syntax with a placeholder: <img src="cid:my-troll-logo"> And then, with the file fields at the bottom, upload the desired image assigning the very same placeholder (in this case my-troll-logo). Real ContentIDs will be managed for you.

3. Define your defaults if any

Keep in mind that you are defining the future interface for sending them (regarding leela it can be changed at any time, but could break your existing calls): When sending an email any default parameter that is not defined will be mandatory.

4. Save and test your new EmailKind.

Now you can use both "Render test" to check the rendering of the templates in your browser, and "Send test" to actually send the email (yeah, will generate an EmailEntry).

5. Usage

To use your brand new email, connect to your queue system and send a JSON body message like this:

{
    "name": "myproject_myemail_description",
    "language": "es",
    "sender": "from@qdqmedia.com",
    "recipients": ["to@qdqmedia.com", "to2@qdqmedia.com", ...],
    "reply_to": ["reply@qdqmedia.com", "reply2@qdqmedia.com", ...],
    "customer_id": "3838383",
    "subject": "This is the email subject",
    "context": {"first_name": "Troll", ...},
    "send_at": 1434029573, # Unix timestamp, UTC
    "check_url": "http://myservice.qdqmedia.com/canisend/4983/323",
    "attachs": [
        {"filename": "invoice.pdf",
        "content_type": "application/pdf",
        "content": "raw content of the file in encoded in base64"},
         ...],
    "backend": "this-backend",
    "meta_fields": {
      "meta1": "metavalue1",
      ...
    }
}

Remember that the encoding of the above message (the body of the queue message) has to be encoded in utf8. About the parameters:

• sender, recipients, reply_to and subject are treated with the defaults logic explained above.
• customer_id is optional, in case you want to relate the email with a customer or anything else. Useful for future API queries.
• context is optional, only add it if your template is going to use it.
• send_at is optional, you can specify the UTC time at which you want your email to be sent. If not specified, will be send as soon as possible.
• check_url is optional, you can set here an url which will be called by leela (GET) just before sending the email, to check if the email is still needed. The response is expected to be a JSON object with two boolean properties: {"allowed": ..., "delete": ...}. If allowed is true the email will be sent. If delete is true, and allowed is false it will be removed without sending.
• attachs is optional, is an array of objects with the keys filename to set the attachment name, content_type describing its MIME Type and content with the raw content of the file itself. It is mandatory to encode the content in base64 to avoid the bytestring to break the JSON format.
• backend is optional. You can specify the email backend to use to send the entry. For specifics on this, check CUSTOM.md.
• meta_fields is optional. You can specify metadata information, but will only make sense if the backend chosen understands metadata.

The variables available in rendering context are the ones defined in the context param object plus an object called meta, which contains information about the EmailKind, with the attributes:

{
  "id": 42,                              # Id of the EmailKind
  "name": "...",                         # EmailKind name
  "language": "es",
  "template": "...",
  "plain_template": "...",
  "default_subject": "...",
  "default_recipients": "...",
  "default_reply_to": "...",
  "default_context": "{}",               # Do not access this var through meta.
  "default_sender": "..."
}

How to define and use reusable email fragments:

It is possible to define Email Kind Fragments to avoid repeating content on emails (such as headers, footers, and so on). To do so the steps are:

• Create your EmailKindFragment using the admin interface.
• Select it in your EmailKind (using the fragments section).
• Once selected, the content of your fragment will be available in the email context when rendering, therefore you can use it in your EmailKind template using {{ fragments.fragment_name }}.

NOTE: when you modify existing EmailKinds that use images to begin to use fragments you need to be careful if you care about history. If you move images from an EmailKind to an EmailKindFragment and you use the EmailKindFragment, the renders of emails sent before the modification will not find the images as those images were defined in the EmailKind. Thence, if the history is important for you, you will need to keep those images both in the EmailKind and the EmailKindFragment.

How to query about emails

Some entry REST points are available to query about emails. The responses are in JSON format, and contain information about counters and pagination. All of them are grouped under the /api/ path:

• /api/entries/ List all entries.
• /api/entries/4/ Retrieves the entry with id = 4.
• /api/entries/?customer_id=06666666 Retrieves all the entries with the customer_id = 6666666.
• /api/entries/?include_kinds=solweb_contact Retrieves all the entries from the email kinds solweb_contact.
• /api/attachs/ List all attachments.
• /api/attachs/12/ Retrieves the attachment with id = 12.
• /api/legacyentries/3567883/ Retrieves the legacy entries previously stored in CDV with the customer_id = 3567883.

The API is fully browsable, so you can just navigate to /api/ and check all the urls and responses visually.

Spam checking

The system can be configured to detect spam. In the leela/settings/custom.py you can define the setting SPAM_CHECK, a dictionary with EmailKind names as keys, and tuples with function paths as values. For example:

SPAM_CHECK = {
    'my_lovely_email': ('custom.spamchecks.has_href',
                        'custom.spamchecks.above_remote_score')
}

In the example, all the entries from the EmailKinds of name my_lovely_email (in all its languages) will be filtered by the functions has_href and above_remote_score. These functions receive the EmailEntry that is about to be sent and should return either True or False. If any of them returns True, the entry will be classified as spam.

Spam checking should not be necessary in the majority of cases, where systems controlled by developers are the ones that send emails. In this case just don't add it to SPAM_CHECK. If your email is sent as a result of a user form submission, you probably need it.

The project comes with no spam check functions by default, its your responsibility to build your own or plug other ones like SpamAssassin, Mollom, etc. given your needs and the nature of your email.

Setup your environment to work on leela

The project uses docker and docker-compose to set up a development environment. Everything is managed by a Makefile to avoid having to type long commands. Available commands are:

• build: It's the first one to be called, will create all the docker images.
• runserver: Starts the Django development server, forwarded to http://127.0.0.1:8005/ in your host.
• scheduler: Starts the scheduler to listen to the queue system, waiting for schedule email calls.
• shell: Starts a shell inside the app container. Remember to activate the virtualenv at /home/qdqmedia/leela before managing the project.
• test: Runs the tests suite.

Deploying leela

Wherever you deploy it, you should take care of the following entry points. Check the details in the Procfile file:

• Admin website served at /admin/ and API served at /api/
• Queue consumer job, configured in the settings of the project. It consumes messages from an AMPQ system in a blocking way. It is the Django command $ python3 manage.py scheduler.
• Email sender job, continuously checks for new scheduled entries an sends them. It is the Django command $ python3 manage.py send_emails.

Checkout the Procfile to know which processes you need to run.

To achieve some extensibility, the project does override some Django settings at run time. Because of the nature of Python running environments and Django settings, it is discouraged to run leela with multiple sender or scheduler processes. As an asynchronous system, sending latency should not bother you.

But if you need to scale the service, the recommended strategy is to deploy multiple independent leela systems and shard the load, for example by the domain of the email, that consume from different AMPQ queues.

Customizing

For a complete customization guide, please check CUSTOM.md