Deploying Gunicorn

We strongly recommend using Gunicorn behind a proxy server.

Nginx Configuration

Although there are many HTTP proxies available, we strongly advise that you use Nginx. If you choose another proxy server you need to make sure that it buffers slow clients when you use default Gunicorn workers. Without this buffering Gunicorn will be easily susceptible to denial-of-service attacks. You can use Hey to check if your proxy is behaving properly.

An example configuration file for fast clients with Nginx:

nginx.conf

  1. worker_processes 1;
  3. user nobody nogroup;
  4. # 'user nobody nobody;' for systems with 'nobody' as a group instead
  5. error_log  /var/log/nginx/error.log warn;
  6. pid /var/run/nginx.pid;
  8. events {
  9.   worker_connections 1024; # increase if you have lots of clients
  10.   accept_mutex off; # set to 'on' if nginx worker_processes > 1
  11.   # 'use epoll;' to enable for Linux 2.6+
  12.   # 'use kqueue;' to enable for FreeBSD, OSX
  13. }
  15. http {
  16.   include mime.types;
  17.   # fallback in case we can't determine a type
  18.   default_type application/octet-stream;
  19.   access_log /var/log/nginx/access.log combined;
  20.   sendfile on;
  22.   upstream app_server {
  23.     # fail_timeout=0 means we always retry an upstream even if it failed
  24.     # to return a good HTTP response
  26.     # for UNIX domain socket setups
  27.     server unix:/tmp/gunicorn.sock fail_timeout=0;
  29.     # for a TCP configuration
  30.     # server 192.168.0.7:8000 fail_timeout=0;
  31.   }
  33.   server {
  34.     # if no Host match, close the connection to prevent host spoofing
  35.     listen 80 default_server;
  36.     return 444;
  37.   }
  39.   server {
  40.     # use 'listen 80 deferred;' for Linux
  41.     # use 'listen 80 accept_filter=httpready;' for FreeBSD
  42.     listen 80;
  43.     client_max_body_size 4G;
  45.     # set the correct host(s) for your site
  46.     server_name example.com www.example.com;
  48.     keepalive_timeout 5;
  50.     # path for static files
  51.     root /path/to/app/current/public;
  53.     location / {
  54.       # checks for static file, if not found proxy to app
  55.       try_files $uri @proxy_to_app;
  56.     }
  58.     location @proxy_to_app {
  59.       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  60.       proxy_set_header X-Forwarded-Proto $scheme;
  61.       proxy_set_header Host $http_host;
  62.       # we don't want nginx trying to do something clever with
  63.       # redirects, we set the Host: header above already.
  64.       proxy_redirect off;
  65.       proxy_pass http://app_server;
  66.     }
  68.     error_page 500 502 503 504 /500.html;
  69.     location = /500.html {
  70.       root /path/to/app/current/public;
  71.     }
  72.   }
  73. }

If you want to be able to handle streaming request/responses or other fancy features like Comet, Long polling, or Web sockets, you need to turn off the proxy buffering. When you do this you must run with one of the async worker classes.

To turn off buffering, you only need to add proxy_buffering off; to your location block:

  1. ...
  2. location @proxy_to_app {
  3.     proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  4.     proxy_set_header Host $http_host;
  5.     proxy_redirect off;
  6.     proxy_buffering off;
  8.     proxy_pass http://app_server;
  9. }
  10. ...

It is recommended to pass protocol information to Gunicorn. Many web frameworks use this information to generate URLs. Without this information, the application may mistakenly generate ‘http’ URLs in ‘https’ responses, leading to mixed content warnings or broken applications. To configure Nginx to pass an appropriate header, add a proxy_set_header directive to your location block:

  1. ...
  2. proxy_set_header X-Forwarded-Proto $scheme;
  3. ...

If you are running Nginx on a different host than Gunicorn you need to tell Gunicorn to trust the X-Forwarded-* headers sent by Nginx. By default, Gunicorn will only trust these headers if the connection comes from localhost. This is to prevent a malicious client from forging these headers:

  1. $ gunicorn -w 3 --forwarded-allow-ips="10.170.3.217,10.170.3.220" test:app

When the Gunicorn host is completely firewalled from the external network such that all connections come from a trusted proxy (e.g. Heroku) this value can be set to ‘*’. Using this value is potentially dangerous if connections to Gunicorn may come from untrusted proxies or directly from clients since the application may be tricked into serving SSL-only content over an insecure connection.

Gunicorn 19 introduced a breaking change concerning how REMOTE_ADDR is handled. Previous to Gunicorn 19 this was set to the value of X-Forwarded-For if received from a trusted proxy. However, this was not in compliance with RFC 3875 which is why the REMOTE_ADDR is now the IP address of the proxy and not the actual user.

To have access logs indicate the actual user IP when proxied, set access_log_format with a format which includes X-Forwarded-For. For example, this format uses X-Forwarded-For in place of REMOTE_ADDR:

  1. %({x-forwarded-for}i)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"

It is also worth noting that the REMOTE_ADDR will be completely empty if you bind Gunicorn to a UNIX socket and not a TCP host:port tuple.

Using Virtualenv

To serve an app from a Virtualenv it is generally easiest to just install Gunicorn directly into the Virtualenv. This will create a set of Gunicorn scripts for that Virtualenv which can be used to run applications normally.

If you have Virtualenv installed, you should be able to do something like this:

  1. $ mkdir ~/venvs/
  2. $ virtualenv ~/venvs/webapp
  3. $ source ~/venvs/webapp/bin/activate
  4. $ pip install gunicorn
  5. $ deactivate

Then you just need to use one of the three Gunicorn scripts that was installed into ~/venvs/webapp/bin.

Note: You can force the installation of Gunicorn in your Virtualenv by passing -I or --ignore-installed option to pip:

  1. $ source ~/venvs/webapp/bin/activate
  2. $ pip install -I gunicorn

Monitoring

Note

Make sure that when using either of these service monitors you do not enable the Gunicorn’s daemon mode. These monitors expect that the process they launch will be the process they need to monitor. Daemonizing will fork-exec which creates an unmonitored process and generally just confuses the monitor services.

Gaffer

Using Gafferd and gaffer

Gaffer can be used to monitor Gunicorn. A simple configuration is:

  1. [process:gunicorn]
  2. cmd = gunicorn -w 3 test:app
  3. cwd = /path/to/project

Then you can easily manage Gunicorn using Gaffer.

Using a Procfile

Create a Procfile in your project:

  1. gunicorn = gunicorn -w 3 test:app

You can launch any other applications that should be launched at the same time.

Then you can start your Gunicorn application using Gaffer:

  1. gaffer start

If gafferd is launched you can also load your Procfile in it directly:

  1. gaffer load

All your applications will be then supervised by gafferd.

Runit

A popular method for deploying Gunicorn is to have it monitored by runit. Here is an example service definition:

  1. #!/bin/sh
  3. GUNICORN=/usr/local/bin/gunicorn
  4. ROOT=/path/to/project
  5. PID=/var/run/gunicorn.pid
  7. APP=main:application
  9. if [ -f $PID ]; then rm $PID; fi
  11. cd $ROOT
  12. exec $GUNICORN -c $ROOT/gunicorn.conf.py --pid=$PID $APP

Save this as /etc/sv/[app_name]/run, and make it executable (chmod u+x /etc/sv/[app_name]/run). Then run ln -s /etc/sv/[app_name] /etc/service/[app_name]. If runit is installed, Gunicorn should start running automatically as soon as you create the symlink.

If it doesn’t start automatically, run the script directly to troubleshoot.

Supervisor

Another useful tool to monitor and control Gunicorn is Supervisor. A simple configuration is:

  1. [program:gunicorn]
  2. command=/path/to/gunicorn main:application -c /path/to/gunicorn.conf.py
  3. directory=/path/to/project
  4. user=nobody
  5. autostart=true
  6. autorestart=true
  7. redirect_stderr=true

Upstart

Using Gunicorn with upstart is simple. In this example we will run the app “myapp” from a virtualenv. All errors will go to /var/log/upstart/myapp.log.

/etc/init/myapp.conf:

  1. description "myapp"
  3. start on (filesystem)
  4. stop on runlevel [016]
  6. respawn
  7. setuid nobody
  8. setgid nogroup
  9. chdir /path/to/app/directory
  11. exec /path/to/virtualenv/bin/gunicorn myapp:app

Systemd

A tool that is starting to be common on linux systems is Systemd. It is a system services manager that allows for strict process management, resources and permissions control.

Below are configurations files and instructions for using systemd to create a unix socket for incoming Gunicorn requests. Systemd will listen on this socket and start gunicorn automatically in response to traffic. Later in this section are instructions for configuring Nginx to forward web traffic to the newly created unix socket:

/etc/systemd/system/gunicorn.service:

  1. [Unit]
  2. Description=gunicorn daemon
  3. Requires=gunicorn.socket
  4. After=network.target
  6. [Service]
  7. Type=notify
  8. # the specific user that our service will run as
  9. User=someuser
  10. Group=someuser
  11. # another option for an even more restricted service is
  12. # DynamicUser=yes
  13. # see http://0pointer.net/blog/dynamic-users-with-systemd.html
  14. RuntimeDirectory=gunicorn
  15. WorkingDirectory=/home/someuser/applicationroot
  16. ExecStart=/usr/bin/gunicorn applicationname.wsgi
  17. ExecReload=/bin/kill -s HUP $MAINPID
  18. KillMode=mixed
  19. TimeoutStopSec=5
  20. PrivateTmp=true
  22. [Install]
  23. WantedBy=multi-user.target

/etc/systemd/system/gunicorn.socket:

  1. [Unit]
  2. Description=gunicorn socket
  4. [Socket]
  5. ListenStream=/run/gunicorn.sock
  6. # Our service won't need permissions for the socket, since it
  7. # inherits the file descriptor by socket activation
  8. # only the nginx daemon will need access to the socket
  9. SocketUser=www-data
  10. # Optionally restrict the socket permissions even more.
  11. # SocketMode=600
  13. [Install]
  14. WantedBy=sockets.target

Next enable and start the socket (it will autostart at boot too):

  1. systemctl enable --now gunicorn.socket

Now let’s see if the nginx daemon will be able to connect to the socket. Running sudo -u www-data curl --unix-socket /run/gunicorn.sock http, our Gunicorn service will be automatically started and you should see some HTML from your server in the terminal.

Note

systemd employs cgroups to track the processes of a service, so it doesn’t need pid files. In the rare case that you need to find out the service main pid, you can use systemctl show --value -p MainPID gunicorn.service, but if you only want to send a signal an even better option is systemctl kill -s HUP gunicorn.service.

Note

www-data is the default nginx user in debian, other distributions use different users (for example: http or nginx). Check your distro to know what to put for the socket user, and for the sudo command.

You must now configure your web proxy to send traffic to the new Gunicorn socket. Edit your nginx.conf to include the following:

/etc/nginx/nginx.conf:

  1. user www-data;
  2. ...
  3. http {
  4.     server {
  5.         listen          8000;
  6.         server_name     127.0.0.1;
  7.         location / {
  8.             proxy_pass http://unix:/run/gunicorn.sock;
  9.         }
  10.     }
  11. }
  12. ...

Note

The listen and server_name used here are configured for a local machine. In a production server you will most likely listen on port 80, and use your URL as the server_name.

Now make sure you enable the nginx service so it automatically starts at boot:

  1. systemctl enable nginx.service

Either reboot, or start Nginx with the following command:

  1. systemctl start nginx

Now you should be able to test Nginx with Gunicorn by visiting http://127.0.0.1:8000/ in any web browser. Systemd is now set up.

Logging

Logging can be configured by using various flags detailed in the configuration documentation or by creating a logging configuration file. Send the USR1 signal to rotate logs if you are using the logrotate utility:

  1. kill -USR1 $(cat /var/run/gunicorn.pid)

Note

Overriding the LOGGING dictionary requires to set disable_existing_loggers: False to not interfere with the Gunicorn logging.

Warning

Gunicorn error log is here to log errors from Gunicorn, not from another application.