Webserver

This topic describes how to configure Airflow to secure your webserver.

Rendering Airflow UI in a Web Frame from another site

Using Airflow in a web frame is enabled by default. To disable this (and prevent click jacking attacks) set the below:

  1. [webserver]
  2. x_frame_enabled = False

Disable Deployment Exposure Warning

Airflow warns when recent requests are made to /robot.txt. To disable this warning set warn_deployment_exposure to False as below:

  1. [webserver]
  2. warn_deployment_exposure = False

Sensitive Variable fields

Variable values that are deemed “sensitive” based on the variable name will be masked in the UI automatically. See Masking sensitive data for more details.

Web Authentication

By default, Airflow requires users to specify a password prior to login. You can use the following CLI commands to create an account:

  1. # create an admin user
  2. airflow users create \
  3.     --username admin \
  4.     --firstname Peter \
  5.     --lastname Parker \
  6.     --role Admin \
  7.     --email spiderman@superhero.org

To deactivate the authentication and allow users to be identified as Anonymous, the following entry in $AIRFLOW_HOME/webserver_config.py needs to be set with the desired role that the Anonymous user will have by default:

  1. AUTH_ROLE_PUBLIC = 'Admin'

Be sure to checkout API for securing the API.

Note

Airflow uses the config parser of Python. This config parser interpolates ‘%’-signs. Make sure escape any % signs in your config file (but not environment variables) as %%, otherwise Airflow might leak these passwords on a config parser exception to a log.

Password

One of the simplest mechanisms for authentication is requiring users to specify a password before logging in.

Please use command line interface airflow users create to create accounts, or do that in the UI.

Other Methods

Since Airflow 2.0, the default UI is the Flask App Builder RBAC. A webserver_config.py configuration file is automatically generated and can be used to configure the Airflow to support authentication methods like OAuth, OpenID, LDAP, REMOTE_USER.

The default authentication option described in the Web Authentication section is related with the following entry in the $AIRFLOW_HOME/webserver_config.py.

  1. AUTH_TYPE = AUTH_DB

Another way to create users is in the UI login page, allowing user self registration through a “Register” button. The following entries in the $AIRFLOW_HOME/webserver_config.py can be edited to make it possible:

  1. AUTH_USER_REGISTRATION = True
  2. AUTH_USER_REGISTRATION_ROLE = "Desired Role For The Self Registered User"
  3. RECAPTCHA_PRIVATE_KEY = 'private_key'
  4. RECAPTCHA_PUBLIC_KEY = 'public_key'
  6. MAIL_SERVER = 'smtp.gmail.com'
  7. MAIL_USE_TLS = True
  8. MAIL_USERNAME = 'yourappemail@gmail.com'
  9. MAIL_PASSWORD = 'passwordformail'
  10. MAIL_DEFAULT_SENDER = 'sender@gmail.com'

The package Flask-Mail needs to be installed through pip to allow user self registration since it is a feature provided by the framework Flask-AppBuilder.

To support authentication through a third-party provider, the AUTH_TYPE entry needs to be updated with the desired option like OAuth, OpenID, LDAP, and the lines with references for the chosen option need to have the comments removed and configured in the $AIRFLOW_HOME/webserver_config.py.

For more details, please refer to Security section of FAB documentation.

Example using team based Authorization with GitHub OAuth

There are a few steps required in order to use team-based authorization with GitHub OAuth.

  • configure OAuth through the FAB config in webserver_config.py

  • create a custom security manager class and supply it to FAB in webserver_config.py

  • map the roles returned by your security manager class to roles that FAB understands.

Here is an example of what you might have in your webserver_config.py:

  1. from flask_appbuilder.security.manager import AUTH_OAUTH
  2. import os
  4. AUTH_TYPE = AUTH_OAUTH
  5. AUTH_ROLES_SYNC_AT_LOGIN = True  # Checks roles on every login
  6. AUTH_USER_REGISTRATION = True  # allow users who are not already in the FAB DB to register
  7. # Make sure to replace this with the path to your security manager class
  8. FAB_SECURITY_MANAGER_CLASS = "your_module.your_security_manager_class"
  9. AUTH_ROLES_MAPPING = {
  10.     "Viewer": ["Viewer"],
  11.     "Admin": ["Admin"],
  12. }
  13. # If you wish, you can add multiple OAuth providers.
  14. OAUTH_PROVIDERS = [
  15.     {
  16.         "name": "github",
  17.         "icon": "fa-github",
  18.         "token_key": "access_token",
  19.         "remote_app": {
  20.             "client_id": os.getenv("OAUTH_APP_ID"),
  21.             "client_secret": os.getenv("OAUTH_APP_SECRET"),
  22.             "api_base_url": "https://api.github.com",
  23.             "client_kwargs": {"scope": "read:user, read:org"},
  24.             "access_token_url": "https://github.com/login/oauth/access_token",
  25.             "authorize_url": "https://github.com/login/oauth/authorize",
  26.             "request_token_url": None,
  27.         },
  28.     },
  29. ]

Here is an example of defining a custom security manager. This class must be available in Python’s path, and could be defined in webserver_config.py itself if you wish.

  1. from airflow.www.security import AirflowSecurityManager
  2. import logging
  3. from typing import Any, List, Union
  4. import os
  6. log = logging.getLogger(__name__)
  7. log.setLevel(os.getenv("AIRFLOW__LOGGING__FAB_LOGGING_LEVEL", "INFO"))
  9. FAB_ADMIN_ROLE = "Admin"
  10. FAB_VIEWER_ROLE = "Viewer"
  11. FAB_PUBLIC_ROLE = "Public"  # The "Public" role is given no permissions
  12. TEAM_ID_A_FROM_GITHUB = 123  # Replace these with real team IDs for your org
  13. TEAM_ID_B_FROM_GITHUB = 456  # Replace these with real team IDs for your org
  16. def team_parser(team_payload: dict[str, Any]) -> list[int]:
  17.     # Parse the team payload from GitHub however you want here.
  18.     return [team["id"] for team in team_payload]
  21. def map_roles(team_list: list[int]) -> list[str]:
  22.     # Associate the team IDs with Roles here.
  23.     # The expected output is a list of roles that FAB will use to Authorize the user.
  25.     team_role_map = {
  26.         TEAM_ID_A_FROM_GITHUB: FAB_ADMIN_ROLE,
  27.         TEAM_ID_B_FROM_GITHUB: FAB_VIEWER_ROLE,
  28.     }
  29.     return list(set(team_role_map.get(team, FAB_PUBLIC_ROLE) for team in team_list))
  32. class GithubTeamAuthorizer(AirflowSecurityManager):
  34.     # In this example, the oauth provider == 'github'.
  35.     # If you ever want to support other providers, see how it is done here:
  36.     # https://github.com/dpgaspar/Flask-AppBuilder/blob/master/flask_appbuilder/security/manager.py#L550
  37.     def get_oauth_user_info(self, provider: str, resp: Any) -> dict[str, Union[str, list[str]]]:
  39.         # Creates the user info payload from Github.
  40.         # The user previously allowed your app to act on their behalf,
  41.         #   so now we can query the user and teams endpoints for their data.
  42.         # Username and team membership are added to the payload and returned to FAB.
  44.         remote_app = self.appbuilder.sm.oauth_remotes[provider]
  45.         me = remote_app.get("user")
  46.         user_data = me.json()
  47.         team_data = remote_app.get("user/teams")
  48.         teams = team_parser(team_data.json())
  49.         roles = map_roles(teams)
  50.         log.debug(f"User info from Github: {user_data}\nTeam info from Github: {teams}")
  51.         return {"username": "github_" + user_data.get("login"), "role_keys": roles}

SSL

SSL can be enabled by providing a certificate and key. Once enabled, be sure to use “https://” in your browser.

  1. [webserver]
  2. web_server_ssl_cert = <path to cert>
  3. web_server_ssl_key = <path to key>

Enabling SSL will not automatically change the web server port. If you want to use the standard port 443, you’ll need to configure that too. Be aware that super user privileges (or cap_net_bind_service on Linux) are required to listen on port 443.

  1. # Optionally, set the server to listen on the standard SSL port.
  2. web_server_port = 443
  3. base_url = http://<hostname or IP>:443

Enable CeleryExecutor with SSL. Ensure you properly generate client and server certs and keys.

  1. [celery]
  2. ssl_active = True
  3. ssl_key = <path to key>
  4. ssl_cert = <path to cert>
  5. ssl_cacert = <path to cacert>

Rate limiting

Airflow can be configured to limit the number of authentication requests in a given time window. We are using Flask-Limiter to achieve that and by default Airflow uses per-webserver default limit of 5 requests per 40 second fixed window. By default no common storage for rate limits is used between the gunicorn processes you run so rate-limit is applied separately for each process, so assuming random distribution of the requests by gunicorn with single webserver instance and default 4 gunicorn workers, the effective rate limit is 5 x 4 = 20 requests per 40 second window (more or less). However you can configure the rate limit to be shared between the processes by using rate limit storage via setting the RATELIMIT_* configuration settings in webserver_config.py. For example, to use Redis as a rate limit storage you can use the following configuration (you need to set redis_host to your Redis instance)

` RATELIMIT_STORAGE_URI = 'redis://redis_host:6379/0 `

You can also configure other rate limit settings in webserver_config.py - for more details, see the Flask Limiter rate limit configuration.