Using Celery With Flask

Working with Flask and Celery

The integration of Celery with Flask is so simple that no extension is required. A Flask application that uses Celery needs to initialize the Celery client as follows:

<span class="kwd">from</span><span class="pln"> flask </span><span class="kwd">import</span> <span class="typ">Flask</span>
<span class="kwd">from</span><span class="pln"> celery </span><span class="kwd">import</span> <span class="typ">Celery</span><span class="pln">

app </span><span class="pun">=</span> <span class="typ">Flask</span><span class="pun">(</span><span class="pln">__name__</span><span class="pun">)</span><span class="pln">
app</span><span class="pun">.</span><span class="pln">config</span><span class="pun">[</span><span class="str">'CELERY_BROKER_URL'</span><span class="pun">]</span> <span class="pun">=</span> <span class="str">'redis://localhost:6379/0'</span><span class="pln">
app</span><span class="pun">.</span><span class="pln">config</span><span class="pun">[</span><span class="str">'CELERY_RESULT_BACKEND'</span><span class="pun">]</span> <span class="pun">=</span> <span class="str">'redis://localhost:6379/0'</span><span class="pln">

celery </span><span class="pun">=</span> <span class="typ">Celery</span><span class="pun">(</span><span class="pln">app</span><span class="pun">.</span><span class="pln">name</span><span class="pun">,</span><span class="pln"> broker</span><span class="pun">=</span><span class="pln">app</span><span class="pun">.</span><span class="pln">config</span><span class="pun">[</span><span class="str">'CELERY_BROKER_URL'</span><span class="pun">])</span><span class="pln">
celery</span><span class="pun">.</span><span class="pln">conf</span><span class="pun">.</span><span class="pln">update</span><span class="pun">(</span><span class="pln">app</span><span class="pun">.</span><span class="pln">config</span><span class="pun">)</span>

As you can see, Celery is initialized by creating an object of class Celery, and passing the application name and the connection URL for the message broker, which I put in app.config under key CELERY_BROKER_URL. This URL tells Celery where the broker service is running. If you run something other than Redis, or have the broker on a different machine, then you will need to change the URL accordingly.

Flask Restful: Project Structure

There are many different ways to organize your Flask-RESTful app, but here we’ll describe one that scales pretty well with larger apps and maintains a nice level organization.

The basic idea is to split your app into three main parts: the routes, the resources, and any common infrastructure.

Here’s an example directory structure:

myapi/
    __init__.py
    app.py          # this file contains your app and routes
    resources/
        __init__.py
        foo.py      # contains logic for /Foo
        bar.py      # contains logic for /Bar
    common/
        __init__.py
        util.py     # just some common infrastructure

The common directory would probably just contain a set of helper functions to fulfill common needs across your application. It could also contain, for example, any custom input/output types your resources need to get the job done.

In the resource files, you just have your resource objects. So here’s what <span class="pre">foo.py</span> might look like:

from flask_restful import Resource

class Foo(Resource):
    def get(self):
        pass
    def post(self):
        pass

The key to this setup lies in <span class="pre">app.py</span>:

from flask import Flask
from flask_restful import Api
from myapi.resources.foo import Foo
from myapi.resources.bar import Bar
from myapi.resources.baz import Baz

app = Flask(__name__)
api = Api(app)

api.add_resource(Foo, '/Foo', '/Foo/<string:id>')
api.add_resource(Bar, '/Bar', '/Bar/<string:id>')
api.add_resource(Baz, '/Baz', '/Baz/<string:id>')

Flask: PyCharm Integration

Prior to PyCharm 2018.1, the Flask CLI features weren’t yet fully integrated into PyCharm. We have to do a few tweaks to get them working smoothly. These instructions should be similar for any other IDE you might want to use.

In PyCharm, with your project open, click on Run from the menu bar and go to Edit Configurations. You’ll be greeted by a screen similar to this: