Reference for Database Migrations

Reference for Database Migrations

Here’s the list of all the Database Migrations that are executed via when you run airflow db upgrade.

Warning

Those migration details are mostly used here to make the users aware when and what kind of migrations will be executed during migrations between specific Airflow versions. The intention here is that the “DB conscious” users might perform an analysis on the migrations and draw conclusions about the impact of the migrations on their Airflow database. Those users might also want to take a look at the ERD Schema of the Database document to understand how the internal DB of Airflow structure looks like. However, you should be aware that the structure is internal and you should not access the DB directly to retrieve or modify any data - you should use the REST API to do that instead.

Revision ID	Revises ID	Airflow Version	Description
`98ae134e6fff` (head)	`6abdffdd4815`	`2.6.0`	Increase length of user identifier columns in `abuser` and `ab_register_user` tables
`6abdffdd4815`	`290244fb8b83`	`2.6.0`	add dttm index on log table
`290244fb8b83`	`1986afd32c1b`	`2.5.0`	Add is_orphaned to DatasetModel
`1986afd32c1b`	`ee8d93fcc81e`	`2.5.0`	Add DagRunNote and TaskInstanceNote
`ee8d93fcc81e`	`e07f49787c9d`	`2.5.0`	Add updated_at column to DagRun and TaskInstance
`e07f49787c9d`	`b0d31815b5a6`	`2.4.3`	Add case-insensitive unique constraint for username
`b0d31815b5a6`	`ecb43d2a1842`	`2.4.2`	Add missing auto-increment to columns on FAB tables
`ecb43d2a1842`	`1486deb605b4`	`2.4.0`	Add processor_subdir column to DagModel, SerializedDagModel and CallbackRequest tables.
`1486deb605b4`	`f4ff391becb5`	`2.4.0`	add dag_owner_attributes table
`f4ff391becb5`	`0038cd0c28b4`	`2.4.0`	Remove smart sensors
`0038cd0c28b4`	`44b7034f6bdc`	`2.4.0`	Add Dataset model
`44b7034f6bdc`	`424117c37d18`	`2.4.0`	compare types between ORM and DB.
`424117c37d18`	`f5fcbda3e651`	`2.4.0`	Add DagWarning model
`f5fcbda3e651`	`3c94c427fdf6`	`2.3.3`	Add indexes for CASCADE deletes on task_instance
`3c94c427fdf6`	`1de7bc13c950`	`2.3.2`	Add cascade to dag_tag foreign key
`1de7bc13c950`	`b1b348e02d07`	`2.3.1`	Add index for `event` column in `log` table.
`b1b348e02d07`	`75d5ed6c2b43`	`2.3.0`	Update dag.default_view to grid.
`75d5ed6c2b43`	`909884dea523`	`2.3.0`	Add map_index to Log.
`909884dea523`	`48925b2719cb`	`2.3.0`	Update migration for FAB tables to add missing constraints
`48925b2719cb`	`4eaab2fe6582`	`2.3.0`	Add map_index to TaskFail
`4eaab2fe6582`	`c97c2ab6aa23`	`2.3.0`	Migrate RTIF to use run_id and map_index
`c97c2ab6aa23`	`c306b5b5ae4a`	`2.3.0`	add callback request table
`c306b5b5ae4a`	`a3bcd0914482`	`2.3.0`	Switch XCom table to use `run_id` and add `map_index`.
`a3bcd0914482`	`e655c0453f75`	`2.3.0`	add data_compressed to serialized_dag
`e655c0453f75`	`f9da662e7089`	`2.3.0`	Add `map_index` column to TaskInstance to identify task- mapping, and a `task_map` table to track mapping values from XCom.
`f9da662e7089`	`786e3737b18f`	`2.3.0`	Add `LogTemplate` table to track changes to config values `log_filename_template`
`786e3737b18f`	`5e3ec427fdd3`	`2.3.0`	Add `timetable_description` column to DagModel for UI.
`5e3ec427fdd3`	`587bdf053233`	`2.3.0`	Increase length of email and username in `ab_user` and `ab_register_user` table to `256` characters
`587bdf053233`	`c381b21cb7e4`	`2.2.4`	Add index for `dag_id` column in `job` table.
`c381b21cb7e4`	`be2bfac3da23`	`2.2.4`	Create a `session` table to store web session data
`be2bfac3da23`	`7b2661a43ba3`	`2.2.3`	Add has_import_errors column to DagModel
`7b2661a43ba3`	`142555e44c17`	`2.2.0`	Change `TaskInstance` and `TaskReschedule` tables from execution_date to run_id.
`142555e44c17`	`54bebd308c5f`	`2.2.0`	Add data_interval[start\|end] to DagModel and DagRun.
`54bebd308c5f`	`30867afad44a`	`2.2.0`	Adds `trigger` table and deferrable operator columns to task instance
`30867afad44a`	`e9304a3141f0`	`2.2.0`	Rename `concurrency` column in `dag` table to`max_active_tasks`
`e9304a3141f0`	`83f031fd9f1c`	`2.2.0`	Make XCom primary key columns non-nullable
`83f031fd9f1c`	`ccde3e26fe78`	`2.2.0`	Improve MSSQL compatibility
`ccde3e26fe78`	`092435bf5d12`	`2.1.4`	Add index on state, dag_id for queued `dagrun`
`092435bf5d12`	`97cdd93827b8`	`2.1.4`	Add `max_active_runs` column to `dag_model` table
`97cdd93827b8`	`a13f7613ad25`	`2.1.3`	Add `queued_at` column in `dag_run` table
`a13f7613ad25`	`e165e7455d70`	`2.1.0`	Resource based permissions for default `Flask-AppBuilder` views
`e165e7455d70`	`90d1635d7b86`	`2.1.0`	Add description field to `Variable` model
`90d1635d7b86`	`2e42bb497a22`	`2.1.0`	Increase maximum length of pool name in `task_instance` table to `256` characters
`2e42bb497a22`	`8646922c8a04`	`2.0.2`	Rename `last_scheduler_run` column in `DAG` table to `last_parsed_time`
`8646922c8a04`	`449b4072c2da`	`2.0.2`	Change default `pool_slots` to `1`
`449b4072c2da`	`82b7c48c147f`	`2.0.2`	Increase size of `connection.extra` field to handle multiple RSA keys
`82b7c48c147f`	`e959f08ac86c`	`2.0.1`	Remove `can_read` permission on config resource for `User` and `Viewer` role
`e959f08ac86c`	`64a7d6477aae`	`2.0.0`	Change field in `DagCode` to `MEDIUMTEXT` for MySql
`64a7d6477aae`	`61ec73d9401f`	`2.0.0`	Fix description field in `connection` to be `text`
`61ec73d9401f`	`2c6edca13270`	`2.0.0`	Add description field to `connection` table
`2c6edca13270`	`849da589634d`	`2.0.0`	Resource based permissions.
`849da589634d`	`45ba3f1493b9`	`2.0.0`	Prefix DAG permissions.
`45ba3f1493b9`	`364159666cbd`	`2.0.0`	add-k8s-yaml-to-rendered-templates
`364159666cbd`	`52d53670a240`	`2.0.0`	Add `creating_job_id` to `DagRun` table
`52d53670a240`	`98271e7606e2`	`2.0.0`	fix_mssql_exec_date_rendered_task_instance_fields_for_MSSQL
`98271e7606e2`	`bef4f3d11e8b`	`2.0.0`	Add `scheduling_decision` to `DagRun` and `DAG`
`bef4f3d11e8b`	`e1a11ece99cc`	`2.0.0`	Drop `KubeResourceVersion` and `KubeWorkerId`
`e1a11ece99cc`	`b247b1e3d1ed`	`2.0.0`	Add external executor ID to TI
`b247b1e3d1ed`	`e38be357a868`	`2.0.0`	Add queued by Job ID to TI
`e38be357a868`	`8d48763f6d53`	`2.0.0`	Add `sensor_instance` table
`8d48763f6d53`	`8f966b9c467a`	`2.0.0`	Add unique constraint to `conn_id`
`8f966b9c467a`	`3c20cacc0044`	`2.0.0`	Set `conn_type` as non-nullable
`3c20cacc0044`	`b25a55525161`	`2.0.0`	Add `run_type` column in `dag_run` table
`b25a55525161`	`bbf4a7ad0465`	`2.0.0`	Increase length of pool name
`bbf4a7ad0465`	`cf5dc11e79ad`	`2.0.0`	Remove id column from xcom
`cf5dc11e79ad`	`03afc6b6f902`	`2.0.0`	Drop `user` and `chart` table
`03afc6b6f902`	`92c57b58940d`	`1.10.13`	Increase length of `Flask-AppBuilder` `ab_view_menu.name` column
`92c57b58940d`	`da3f683c3a5a`	`1.10.13`	Create FAB Tables
`da3f683c3a5a`	`a66efa278eea`	`1.10.12`	Add `dag_hash` Column to `serialized_dag` table
`a66efa278eea`	`952da73b5eff`	`1.10.11`	Add Precision to `execution_date` in `RenderedTaskInstanceFields` table
`952da73b5eff`	`852ae6c715af`	`1.10.10`	Add `dag_code` table
`852ae6c715af`	`a4c2fd67d16b`	`1.10.10`	Add `RenderedTaskInstanceFields` table
`a4c2fd67d16b`	`7939bcff74ba`	`1.10.10`	Add `pool_slots` field to `task_instance`
`7939bcff74ba`	`fe461863935f`	`1.10.8`	Add `DagTags` table
`fe461863935f`	`08364691d074`	`1.10.7`	Increase length for connection password
`08364691d074` (merge_point)	`a56c9515abdc`, `004c1210f153`, `74effc47d867`, `b3b105409875`	`1.10.7`	Straighten out the migrations
`a56c9515abdc`	`c8ffec048a3b`	`1.10.3`	Remove `dag_stat` table
`004c1210f153`	`939bb1e647c8`	`1.10.4`	Increase queue name size limit
`74effc47d867`	`6e96a59344a4`	`1.10.5`	change datetime to datetime2(6) on MSSQL tables.
`b3b105409875`	`d38e04c12aa2`	`1.10.7`	Add `root_dag_id` to `DAG`
`d38e04c12aa2`	`6e96a59344a4`	`1.10.7`	Add `serialized_dag` table
`6e96a59344a4` (branch_point)	`939bb1e647c8`	`1.10.4`	Make `TaskInstance.pool` not nullable
`939bb1e647c8` (branch_point)	`dd4ecb8fbee3`	`1.10.3`	task reschedule foreign key on cascade delete
`dd4ecb8fbee3`	`c8ffec048a3b`	`1.10.3`	Add schedule interval to dag
`c8ffec048a3b` (branch_point)	`41f5f12752f8`	`1.10.3`	Add `description` and `default_view` column to `dag` table
`41f5f12752f8`	`03bc53e68815`	`1.10.2`	Add superuser field
`03bc53e68815` (merge_point)	`0a2a5b66e19d`, `bf00311e1990`	`1.10.2`	Merge migrations Heads.
`0a2a5b66e19d`	`9635ae0956e7`	`1.10.2`	Add `task_reschedule` table
`bf00311e1990`	`dd25f486b8ea`	`1.10.2`	Add index to `task_instance` table
`dd25f486b8ea`	`9635ae0956e7`	`1.10.2`	Add index on `log` table
`9635ae0956e7` (branch_point)	`856955da8476`	`1.10.0`	Create index on `task_fail` table
`856955da8476`	`f23433877c24`	`1.10.0`	Fix Sqlite foreign key
`f23433877c24`	`05f30312d566`	`1.10.0`	Fix MySQL not null constraint
`05f30312d566` (merge_point)	`86770d1215c0`, `0e2a74e0fc9f`	`1.10.0`	Merge migrations Heads
`86770d1215c0`	`27c6a30d7c24`	`1.10.0`	Add kubernetes scheduler uniqueness
`27c6a30d7c24`	`33ae817a1ff4`	`1.10.0`	Add `executor_config` column to `task_instance` table
`33ae817a1ff4`	`d2ae31099d61`	`1.10.0`	Add Kubernetes resource check-pointing
`0e2a74e0fc9f`	`d2ae31099d61`	`1.10.0`	Add time zone awareness
`d2ae31099d61` (branch_point)	`947454bf1dff`	`1.8.2`	Increase text size for MySQL (not relevant for other DBs’ text types)
`947454bf1dff`	`bdaa763e6c56`	`1.8.2`	Create index on `job_id` column in `task_instance` table
`bdaa763e6c56`	`cc1e65623dc7`	`1.8.2`	Make xcom value column a large binary
`cc1e65623dc7`	`127d2bf2dfa7`	`1.8.2`	Add `max_tries` column to `task_instance`
`127d2bf2dfa7`	`5e7d17757c7a`	`1.7.1.3`	Add `dag_id`/`state` index on `dag_run` table
`5e7d17757c7a`	`8504051e801b`	`1.7.1.3`	Add `pid` field to `TaskInstance`
`8504051e801b`	`4addfa1236f1`	`1.7.1.3`	Add indices on `xcom` table
`4addfa1236f1`	`f2ca10b85618`	`1.7.1.3`	Add fractional seconds to MySQL tables
`f2ca10b85618`	`64de9cddf6c9`	`1.7.1.3`	Add `dag_stats` table
`64de9cddf6c9`	`211e584da130`	`1.7.1.3`	Add `task_fail` table
`211e584da130`	`2e82aab8ef20`	`1.7.1.3`	Add TI state index
`2e82aab8ef20`	`1968acfc09e3`	`1.7.1`	Rename user table
`1968acfc09e3`	`bba5a7cfc896`	`1.7.0`	Add `is_encrypted` column to variable table
`bba5a7cfc896`	`bbc73705a13e`	`1.7.0`	Add a column to track the encryption state of the ‘Extra’ field in connection
`bbc73705a13e`	`4446e08588`	`1.7.0`	Add `notification_sent` column to `sla_miss` table
`4446e08588`	`561833c1c74b`	`1.6.2`	Add `start_date` and `end_date` in `dag_run` table
`561833c1c74b`	`40e67319e3a9`	`1.6.2`	Add `password` column to `user` table
`40e67319e3a9`	`2e541a1dcfed`	`1.6.0`	Add `conf` column in `dag_run` table
`2e541a1dcfed`	`1b38cef5b76e`	`1.6.0`	Change `task_instance.task_duration` type to `FLOAT`
`1b38cef5b76e`	`502898887f84`	`1.6.0`	Add `dag_run` table
`502898887f84`	`52d714495f0`	`1.6.0`	Adding `extra` column to `Log` table
`52d714495f0`	`338e90f54d61`	`1.5.2`	Add indices in `job` table
`338e90f54d61`	`13eb55f81627`	`1.5.0`	Add `operator` and `queued_dttm` to `task_instance` table
`13eb55f81627`	`1507a7289a2f`	`1.5.0`	Maintain history for compatibility with earlier migrations
`1507a7289a2f`	`e3a246e0dc1`	`1.5.0`	Add `is_encrypted` column in `connection` table
`e3a246e0dc1` (base)	`None`	`1.5.0`	current schema