TiDB 5.0.4 Release Notes

Release Date: September 27, 2021

TiDB version: 5.0.4

Compatibility changes

  • TiDB

    • Fix the issue that executing SHOW VARIABLES in a new session is slow. This fix reverts some changes made in #19341 and might cause compatibility issues. #24326

    • Change the default value of the tidb_stmt_summary_max_stmt_count variable from 200 to 3000 #25873

    • The following bug fixes change execution results, which might cause upgrade incompatibilities:

      • Fix the issue that TiDB returns wrong result when the children of UNION contain the NULL value #26559
      • Fix the issue that greatest(datetime) union null returns empty string #26532
      • Fix the issue that the behavior of the last_day function is incompatible in SQL mode #26000
      • Fix the issue that the having clause might not work correctly #26496
      • Fix the wrong execution results that occur when the collations around the between expression are different #27146
      • Fix the wrong execution results that occur when the column in the group_concat function has a non-bin collation #27429
      • Fix an issue that using a count(distinct) expression on multiple columns returns wrong result when the new collation is enabled #27091
      • Fix the result wrong that occurs when the argument of the extract function is a negative duration #27236
      • Fix the issue that inserting an invalid date does not report an error when the SQL_MODE is ‘STRICT_TRANS_TABLES’ #26762
      • Fix the issue that using an invalid default date does not report an error when the SQL_MODE is ‘NO_ZERO_IN_DATE’ #26766
      • Fix a bug on the query range of prefix index #26029
      • Fix the issue that the LOAD DATA statement might abnormally import non-utf8 data #25979
      • Fix the issue that insert ignore on duplicate update might insert wrong data when the secondary index has the same column with the primary key #25809
      • Fix the issue that insert ignore duplicate update might insert wrong data when a partitioned table has a clustered index #25846
      • Fix the issue that the query result might be wrong when the key is the ENUM type in point get or batch point get #24562
      • Fix the wrong result that occurs when dividing a BIT-type value #23479
      • Fix the issue that the results of prepared statements and direct queries might be inconsistent #22949
      • Fix the issue that the query result might be wrong when a YEAR type is compared with a string or an integer type #23262

Feature enhancements

  • TiDB

    • Support setting tidb_enforce_mpp=1 to ignore the optimizer estimation and forcibly use the MPP mode #26382

  • TiKV

    • Support changing TiCDC configurations dynamically #10645

  • PD

    • Add OIDC-based SSO support for TiDB Dashboard #3884

  • TiFlash

    • Support the HAVING() function in DAG requests
    • Support the DATE() function
    • Add Grafana panels for write throughput per instance

Improvements

  • TiDB

    • Trigger auto-analyze based on the histogram row count #24237
    • Stop sending requests to a TiFlash node for a period if the node has failed and restarted before #26757
    • Increase the split region upper limit to make split table and presplit more stable #26657
    • Support retry for MPP queries #26483
    • Check the availability of TiFlash before launching MPP queries #1807
    • Support the stable result mode to make the query result more stable #26084
    • Support the MySQL system variable init_connect and its associated features #18894
    • Thoroughly push down the COUNT(DISTINCT) aggregation function in the MPP mode #25861
    • Print log warnings when the aggregation function cannot be pushed down in EXPLAIN statements #25736
    • Add error labels for TiFlashQueryTotalCounter in Grafana dashboard #25327
    • Support getting the MVCC data of a clustered index table through a secondary index by HTTP API #24209
    • Optimize the memory allocation of prepared statement in parser #24371

  • TiKV

    • Handle read ready and write ready separately to reduce read latency #10475
    • Reduce the size of Resolved TS messages to save network bandwidth #2448
    • Drop log instead of blocking threads when the slogger thread is overloaded and the queue is filled up #10841
    • Make the slow log of TiKV coprocessor only consider the time spent on processing requests #10841
    • Make prewrite as idempotent as possible to reduce the chance of undetermined errors #10587
    • Avoid the false “GC can not work” alert under low write flow #10662
    • Make the database to be restored always match the original cluster size during backup. #10643
    • Ensure that the panic output is flushed to the log #9955

  • PD

    • Improve the performance of synchronizing Region information between PDs #3993

  • Tools

    • Dumpling

      • Support backing up MySQL-compatible databases that do not support the START TRANSACTION ... WITH CONSISTENT SNAPSHOT or the SHOW CREATE TABLE syntax #309

    • TiCDC

      • Optimize memory management when Unified Sorter is using memory to sort #2553
      • Prohibit operating TiCDC clusters across major or minor versions #2598
      • Reduce the goroutine usage when a table’s Regions are all transferred away from a TiKV node #2284
      • Remove file sorter #2326
      • Always pull the old values from TiKV and the output is adjusted according to enable-old-value #2301
      • Improve the error message returned when a PD endpoint misses the certificate #1973
      • Optimize workerpool for fewer goroutines when concurrency is high #2211
      • Add a global gRPC connection pool and share gRPC connections among KV clients #2533

Bug Fixes

  • TiDB

    • Fix the issue that TiDB might panic when querying a partitioned table and the partition key has the IS NULL condition #23802
    • Fix the issue that the overflow check of the FLOAT64 type is different with that of MySQL #23897
    • Fix the wrong character set and collation for the case when expression #26662
    • Fix the issue that committing pessimistic transactions might cause write conflicts #25964
    • Fix a bug that the index keys in a pessimistic transaction might be repeatedly committed #26359 #10600
    • Fix the issue that TiDB might panic when resolving the async commit locks #25778
    • Fix a bug that a column might not be found when using INDEX MERGE #25045
    • Fix a bug that ALTER USER REQUIRE SSL clears users’ authentication_string #25225
    • Fix a bug that the value of the tidb_gc_scan_lock_mode global variable on a new cluster shows “PHYSICAL” instead of the actual default mode “LEGACY” #25100
    • Fix the bug that the TIKV_REGION_PEERS system table does not show the correct DOWN status #24879
    • Fix the issue of memory leaks that occurs when HTTP API is used #24649
    • Fix the issue that views do not support DEFINER #24414
    • Fix the issue that tidb-server --help exits with the code 2 #24046
    • Fix the issue that setting the global variable dml_batch_size does not take effect #24709
    • Fix the issue that using read_from_storage and partitioned table at the same time causes an error #20372
    • Fix the issue that TiDB panics when executing the projection operator #24264
    • Fix the issue that statistics might cause queries to panic #24061
    • Fix the issue that using the approx_percentile function on a BIT column might panic #23662
    • Fix the issue that the metrics on the Coprocessor Cache panel in Grafana are wrong #26338
    • Fix the issue that concurrently truncating the same partition causes DDL statements to stuck #26229
    • Fix the issue of wrong query results that occurs when the session variable is used as the GROUP BY item #27106
    • Fix the wrong implicit conversion between VARCHAR and timestamp when joining tables #25902
    • Fix the wrong results in associated subquery statements #27233

  • TiKV

    • Fix the potential disk full issue caused by corrupted snapshot files #10813
    • Fix the TiKV panic issue that occurs when upgrading from a pre-5.0 version with Titan enabled #10843
    • Fix the issue that TiKV of a newer version cannot be rolled back to v5.0.x #10843
    • Fix the TiKV panic issue that occurs when upgrading from a pre-5.0 version to a 5.0 version or later. If a cluster was upgraded from TiKV v3.x with Titan enabled before the upgrade, this cluster might encounter the issue. #10774
    • Fix the parsing failure caused by the left pessimistic locks #26404
    • Fix the panic that occurs when calculating duration on certain platforms #10571
    • Fix the issue that the keys of batch_get_command in Load Base Split are unencoded #10542

  • PD

    • Fix the issue that PD does not fix the down peers in time #4077
    • Fix the issue that the replica count of the default placement rules stays constant after replication.max-replicas is updated #3886
    • Fix a bug that PD might panic when scaling out TiKV #3868
    • Fix the scheduling conflict issue that occurs when multiple schedulers are running at same time #3807
    • Fix the issue that the scheduler might appear again even if it has been deleted #2572

  • TiFlash

    • Fix the potential panic issue that occurs when running table scan tasks
    • Fix the potential memory leak issue that occurs when executing MPP tasks
    • Fix a bug that TiFlash raises the duplicated region error when handling DAQ requests
    • Fix the issue of unexpected results when executing the aggregation functions COUNT or COUNT DISTINCT
    • Fix the potential panic issue that occurs when executing MPP tasks
    • Fix a potential bug that TiFlash cannot restore data when deployed on multiple disks
    • Fix the potential panic issue that occurs when deconstructing SharedQueryBlockInputStream
    • Fix the potential panic issue that occurs when deconstructing MPPTask
    • Fix the issue of unexpected results when TiFlash fails to establish MPP connections
    • Fix the potential panic issue that occurs when resolving locks
    • Fix the issue that the store size in metrics is inaccurate under heavy writing
    • Fix a bug of incorrect results that occurs when queries contain filters like CONSTANT, <, <=, >, >=, or COLUMN
    • Fix the potential issue that TiFlash cannot garbage-collect the delta data after running for a long time
    • Fix a potential bug that metrics display wrong values
    • Fix the potential issue of data inconsistency that occurs when TiFlash is deployed on multiple disks

  • Tools

    • Dumpling

      • Fix the issue that the execution of show table status is stuck in MySQL 8.0.3 or a later version #322

    • TiCDC

      • Fix the issue of process panic that occurs when encoding the data types such as mysql.TypeString, mysql.TypeVarString, mysql.TypeVarchar into JSON #2758
      • Fix a data inconsistency issue that occurs because multiple processors might write data to the same table when this table is being re-scheduled #2417
      • Decrease the gRPC window size to avoid the OOM that occurs when TiCDC captures too many Regions #2724
      • Fix the error that the gRPC connection is frequently broken when the memory pressure is high #2202
      • Fix a bug that causes TiCDC to panic on the unsigned TINYINT type #2648
      • Fix the issue that TiCDC Open Protocol outputs an empty value when inserting a transaction and deleting data of the same row in the upstream #2612
      • Fix a bug that DDL handling fails when a changefeed starts at the finish TS of a schema change #2603
      • Fix the issue that irresponsive downstreams interrupt the replication task in old owner until the task times out #2295
      • Fix a bug in metadata management #2558
      • Fix the issue of data inconsistency that occurs after the TiCDC owner switch #2230
      • Fix the issue that outdated capture might appear in the output of the capture list command #2388
      • Fix the ErrSchemaStorageTableMiss error that occurs when the DDL Job duplication is encountered in the integrated test #2422
      • Fix the bug that a changefeed cannot be removed if the ErrGCTTLExceeded error occurs #2391
      • Fix a bug that replicating large tables to cdclog fails #1259 #2424
      • Fix the CLI backward compatibility issue #2373
      • Fix the issue of insecure concurrent access to the map in SinkManager #2299
      • Fix the issue of potential DDL loss when the owner crashes when executing DDL statements #1260
      • Fix the issue that the lock is resolved immediately after a Region is initialized #2188
      • Fix the issue of extra partition dispatching that occurs when adding a new partitioned table #2263
      • Fix the issue that TiCDC keeps warning on removed changefeeds #2156