Testing Flask Applications

Something that is untested is broken.

The origin of this quote is unknown and while it is not entirely correct, it is also not far from the truth. Untested applications make it hard to improve existing code and developers of untested applications tend to become pretty paranoid. If an application has automated tests, you can safely make changes and instantly know if anything breaks.

Flask provides a way to test your application by exposing the Werkzeug test Client and handling the context locals for you. You can then use that with your favourite testing solution.

In this documentation we will use the pytest package as the base framework for our tests. You can install it with pip, like so:

  1. $ pip install pytest

The Application

First, we need an application to test; we will use the application from the Tutorial. If you don’t have that application yet, get the source code from the examples.

So that we can import the module flaskr correctly, we need to run pip install -e . in the folder tutorial.

The Testing Skeleton

We begin by adding a tests directory under the application root. Then create a Python file to store our tests (test_flaskr.py). When we format the filename like test_*.py, it will be auto-discoverable by pytest.

Next, we create a pytest fixture called client() that configures the application for testing and initializes a new database:

  1. import os
  2. import tempfile
  4. import pytest
  6. from flaskr import create_app
  9. @pytest.fixture
  10. def client():
  11.     db_fd, flaskr.app.config['DATABASE'] = tempfile.mkstemp()
  12.     flaskr.app.config['TESTING'] = True
  14.     with flaskr.app.test_client() as client:
  15.         with flaskr.app.app_context():
  16.             flaskr.init_db()
  17.         yield client
  19.     os.close(db_fd)
  20.     os.unlink(flaskr.app.config['DATABASE'])

This client fixture will be called by each individual test. It gives us a simple interface to the application, where we can trigger test requests to the application. The client will also keep track of cookies for us.

During setup, the TESTING config flag is activated. What this does is disable error catching during request handling, so that you get better error reports when performing test requests against the application.

Because SQLite3 is filesystem-based, we can easily use the tempfile module to create a temporary database and initialize it. The mkstemp() function does two things for us: it returns a low-level file handle and a random file name, the latter we use as database name. We just have to keep the db_fd around so that we can use the os.close() function to close the file.

To delete the database after the test, the fixture closes the file and removes it from the filesystem.

If we now run the test suite, we should see the following output:

  1. $ pytest
  3. ================ test session starts ================
  4. rootdir: ./flask/examples/flaskr, inifile: setup.cfg
  5. collected 0 items
  7. =========== no tests ran in 0.07 seconds ============

Even though it did not run any actual tests, we already know that our flaskr application is syntactically valid, otherwise the import would have died with an exception.

The First Test

Now it’s time to start testing the functionality of the application. Let’s check that the application shows “No entries here so far” if we access the root of the application (/). To do this, we add a new test function to test_flaskr.py, like this:

  1. def test_empty_db(client):
  2.     """Start with a blank database."""
  4.     rv = client.get('/')
  5.     assert b'No entries here so far' in rv.data

Notice that our test functions begin with the word test; this allows pytest to automatically identify the function as a test to run.

By using client.get we can send an HTTP GET request to the application with the given path. The return value will be a response_class object. We can now use the data attribute to inspect the return value (as string) from the application. In this case, we ensure that 'No entries here so far' is part of the output.

Run it again and you should see one passing test:

  1. $ pytest -v
  3. ================ test session starts ================
  4. rootdir: ./flask/examples/flaskr, inifile: setup.cfg
  5. collected 1 items
  7. tests/test_flaskr.py::test_empty_db PASSED
  9. ============= 1 passed in 0.10 seconds ==============

Logging In and Out

The majority of the functionality of our application is only available for the administrative user, so we need a way to log our test client in and out of the application. To do this, we fire some requests to the login and logout pages with the required form data (username and password). And because the login and logout pages redirect, we tell the client to follow_redirects.

Add the following two functions to your test_flaskr.py file:

  1. def login(client, username, password):
  2.     return client.post('/login', data=dict(
  3.         username=username,
  4.         password=password
  5.     ), follow_redirects=True)
  8. def logout(client):
  9.     return client.get('/logout', follow_redirects=True)

Now we can easily test that logging in and out works and that it fails with invalid credentials. Add this new test function:

  1. def test_login_logout(client):
  2.     """Make sure login and logout works."""
  4.     username = flaskr.app.config["USERNAME"]
  5.     password = flaskr.app.config["PASSWORD"]
  7.     rv = login(client, username, password)
  8.     assert b'You were logged in' in rv.data
  10.     rv = logout(client)
  11.     assert b'You were logged out' in rv.data
  13.     rv = login(client, f"{username}x", password)
  14.     assert b'Invalid username' in rv.data
  16.     rv = login(client, username, f'{password}x')
  17.     assert b'Invalid password' in rv.data

Test Adding Messages

We should also test that adding messages works. Add a new test function like this:

  1. def test_messages(client):
  2.     """Test that messages work."""
  4.     login(client, flaskr.app.config['USERNAME'], flaskr.app.config['PASSWORD'])
  5.     rv = client.post('/add', data=dict(
  6.         title='<Hello>',
  7.         text='<strong>HTML</strong> allowed here'
  8.     ), follow_redirects=True)
  9.     assert b'No entries here so far' not in rv.data
  10.     assert b'&lt;Hello&gt;' in rv.data
  11.     assert b'<strong>HTML</strong> allowed here' in rv.data

Here we check that HTML is allowed in the text but not in the title, which is the intended behavior.

Running that should now give us three passing tests:

  1. $ pytest -v
  3. ================ test session starts ================
  4. rootdir: ./flask/examples/flaskr, inifile: setup.cfg
  5. collected 3 items
  7. tests/test_flaskr.py::test_empty_db PASSED
  8. tests/test_flaskr.py::test_login_logout PASSED
  9. tests/test_flaskr.py::test_messages PASSED
  11. ============= 3 passed in 0.23 seconds ==============

Other Testing Tricks

Besides using the test client as shown above, there is also the test_request_context() method that can be used in combination with the with statement to activate a request context temporarily. With this you can access the request, g and session objects like in view functions. Here is a full example that demonstrates this approach:

  1. import flask
  3. app = flask.Flask(__name__)
  5. with app.test_request_context('/?name=Peter'):
  6.     assert flask.request.path == '/'
  7.     assert flask.request.args['name'] == 'Peter'

All the other objects that are context bound can be used in the same way.

If you want to test your application with different configurations and there does not seem to be a good way to do that, consider switching to application factories (see Application Factories).

Note however that if you are using a test request context, the before_request() and after_request() functions are not called automatically. However teardown_request() functions are indeed executed when the test request context leaves the with block. If you do want the before_request() functions to be called as well, you need to call preprocess_request() yourself:

  1. app = flask.Flask(__name__)
  3. with app.test_request_context('/?name=Peter'):
  4.     app.preprocess_request()
  5.     ...

This can be necessary to open database connections or something similar depending on how your application was designed.

If you want to call the after_request() functions you need to call into process_response() which however requires that you pass it a response object:

  1. app = flask.Flask(__name__)
  3. with app.test_request_context('/?name=Peter'):
  4.     resp = Response('...')
  5.     resp = app.process_response(resp)
  6.     ...

This in general is less useful because at that point you can directly start using the test client.

Faking Resources and Context

Changelog

New in version 0.10.

A very common pattern is to store user authorization information and database connections on the application context or the flask.g object. The general pattern for this is to put the object on there on first usage and then to remove it on a teardown. Imagine for instance this code to get the current user:

  1. def get_user():
  2.     user = getattr(g, 'user', None)
  3.     if user is None:
  4.         user = fetch_current_user_from_database()
  5.         g.user = user
  6.     return user

For a test it would be nice to override this user from the outside without having to change some code. This can be accomplished with hooking the flask.appcontext_pushed signal:

  1. from contextlib import contextmanager
  2. from flask import appcontext_pushed, g
  4. @contextmanager
  5. def user_set(app, user):
  6.     def handler(sender, **kwargs):
  7.         g.user = user
  8.     with appcontext_pushed.connected_to(handler, app):
  9.         yield

And then to use it:

  1. from flask import json, jsonify
  3. @app.route('/users/me')
  4. def users_me():
  5.     return jsonify(username=g.user.username)
  7. with user_set(app, my_user):
  8.     with app.test_client() as c:
  9.         resp = c.get('/users/me')
  10.         data = json.loads(resp.data)
  11.         assert data['username'] == my_user.username

Keeping the Context Around

Changelog

New in version 0.4.

Sometimes it is helpful to trigger a regular request but still keep the context around for a little longer so that additional introspection can happen. With Flask 0.4 this is possible by using the test_client() with a with block:

  1. app = flask.Flask(__name__)
  3. with app.test_client() as c:
  4.     rv = c.get('/?tequila=42')
  5.     assert request.args['tequila'] == '42'

If you were to use just the test_client() without the with block, the assert would fail with an error because request is no longer available (because you are trying to use it outside of the actual request).

Accessing and Modifying Sessions

Changelog

New in version 0.8.

Sometimes it can be very helpful to access or modify the sessions from the test client. Generally there are two ways for this. If you just want to ensure that a session has certain keys set to certain values you can just keep the context around and access flask.session:

  1. with app.test_client() as c:
  2.     rv = c.get('/')
  3.     assert flask.session['foo'] == 42

This however does not make it possible to also modify the session or to access the session before a request was fired. Starting with Flask 0.8 we provide a so called “session transaction” which simulates the appropriate calls to open a session in the context of the test client and to modify it. At the end of the transaction the session is stored and ready to be used by the test client. This works independently of the session backend used:

  1. with app.test_client() as c:
  2.     with c.session_transaction() as sess:
  3.         sess['a_key'] = 'a value'
  5.     # once this is reached the session was stored and ready to be used by the client
  6.     c.get(...)

Note that in this case you have to use the sess object instead of the flask.session proxy. The object however itself will provide the same interface.

Testing JSON APIs

Changelog

New in version 1.0.

Flask has great support for JSON, and is a popular choice for building JSON APIs. Making requests with JSON data and examining JSON data in responses is very convenient:

  1. from flask import request, jsonify
  3. @app.route('/api/auth')
  4. def auth():
  5.     json_data = request.get_json()
  6.     email = json_data['email']
  7.     password = json_data['password']
  8.     return jsonify(token=generate_token(email, password))
  10. with app.test_client() as c:
  11.     rv = c.post('/api/auth', json={
  12.         'email': 'flask@example.com', 'password': 'secret'
  13.     })
  14.     json_data = rv.get_json()
  15.     assert verify_token(email, json_data['token'])

Passing the json argument in the test client methods sets the request data to the JSON-serialized object and sets the content type to application/json. You can get the JSON data from the request or response with get_json.

Testing CLI Commands

Click comes with utilities for testing your CLI commands. A CliRunner runs commands in isolation and captures the output in a Result object.

Flask provides test_cli_runner() to create a FlaskCliRunner that passes the Flask app to the CLI automatically. Use its invoke() method to call commands in the same way they would be called from the command line.

  1. import click
  3. @app.cli.command('hello')
  4. @click.option('--name', default='World')
  5. def hello_command(name):
  6.     click.echo(f'Hello, {name}!')
  8. def test_hello():
  9.     runner = app.test_cli_runner()
  11.     # invoke the command directly
  12.     result = runner.invoke(hello_command, ['--name', 'Flask'])
  13.     assert 'Hello, Flask' in result.output
  15.     # or by name
  16.     result = runner.invoke(args=['hello'])
  17.     assert 'World' in result.output

In the example above, invoking the command by name is useful because it verifies that the command was correctly registered with the app.

If you want to test how your command parses parameters, without running the command, use its make_context() method. This is useful for testing complex validation rules and custom types.

  1. def upper(ctx, param, value):
  2.     if value is not None:
  3.         return value.upper()
  5. @app.cli.command('hello')
  6. @click.option('--name', default='World', callback=upper)
  7. def hello_command(name):
  8.     click.echo(f'Hello, {name}!')
  10. def test_hello_params():
  11.     context = hello_command.make_context('hello', ['--name', 'flask'])
  12.     assert context.params['name'] == 'FLASK'