TiDB 4.0.14 Release Notes

Release date: July 27, 2021

TiDB version: 4.0.14

Compatibility changes

  • TiDB

    • Change the default value of tidb_multi_statement_mode from WARN to OFF in v4.0. It is recommended to use the multi-statement feature of your client library instead. See the documentation on tidb_multi_statement_mode for details. #25749
    • Upgrade Grafana dashboard from v6.1.16 to v7.5.7 to solve two security vulnerabilities. See the Grafana blog post for details.
    • Change the default value of the tidb_stmt_summary_max_stmt_count variable from 200 to 3000 #25872

  • TiKV

    • Change the default value of merge-check-tick-interval from 10 to 2 to speed up the Region merge process #9676

Feature enhancements

  • TiKV

    • Add a metric pending to monitor the number of pending PD heartbeats, which helps locate the issue of slow PD threads #10008
    • Support using the virtual-host addressing mode to make BR support the S3-compatible storage #10242

  • TiDB Dashboard

    • Support OIDC SSO. By setting the OIDC-compatible SSO services (such as Okta and Auth0), users can log into TiDB Dashboard without entering the SQL password. #960
    • Add the Debug API UI, which is an alternative method to the command line to call several common TiDB and PD internal APIs for advanced debugging #927

Improvements

  • TiDB

    • Change the LOCK record into the PUT record for the index keys using point get or batch point get for UPDATE reads #26223
    • Support the MySQL system variable init_connect and its associated features #26031
    • Support the stable result mode to make the query results more stable #26003
    • Support pushing down the built-in function json_unquote() to TiKV #25721
    • Make the SQL Plan Management (SPM) not affected by the character set #23295

  • TiKV

    • Shutdown the status server first to make sure that the client can correctly check the shutdown status #10504
    • Always respond to stale peers to make sure that these peers are cleared quicker #10400
    • Limit the TiCDC sink’s memory consumption #10147
    • When a Region is too large, use the even split to speed up the split process #10275

  • PD

    • Reduce the conflicts among multiple schedulers that run at the same time #3858 #3854

  • TiDB Dashboard

    • Update TiDB Dashboard to v2021.07.17.1 #3882
    • Support sharing the current session as a read-only session to avoid further modification to it #960

  • Tools

    • Backup & Restore (BR)

      • Speed up restore by merging small backup files #655

    • Dumpling

      • Always split tables using _tidb_rowid when the upstream is a TiDB v3.x cluster, which helps reduce TiDB’s memory usage #306

    • TiCDC

      • Improve the error message returned when a PD endpoint misses the certificate #1973
      • Make the sorter I/O errors more user-friendly #1976
      • Add a concurrency limit on the Region incremental scan in the KV client to reduce the pressure of TiKV #1926
      • Add metrics for the table memory consumption #1884
      • Add capture-session-ttl to the TiCDC server configuration #2169

Bug fixes

  • TiDB

    • Fix the issue that the SELECT result is incompatible with MySQL when joining a subquery with a WHERE clause evaluated to false #24865
    • Fix the calculation error of the ifnull function that occurs when the argument is the ENUM or SET type #24944
    • Fix the wrong aggregate pruning in some cases #25202
    • Fix the incorrect result of the merge join operation that might occur when the column is the SET type #25669
    • Fix the issue that TiDB returns wrong results for cartesian join #25591
    • Fix the panic issue that occurs when SELECT ... FOR UPDATE works on a join operation and the join uses a partitioned table #20028
    • Fix the issue that the cached prepared plan is incorrectly used for point get #24741
    • Fix the issue that the LOAD DATA statement can abnormally import non-utf8 data #25979
    • Fix a potential memory leak issue that occurs when accessing the statistics via an HTTP API #24650
    • Fix a security issue that occurs when executing the ALTER USER statement #25225
    • Fix a bug that the TIKV_REGION_PEERS table cannot correctly handle the DOWN status #24879
    • Fix the issue that invalid strings are not truncated when parsing DateTime #22231
    • Fix the issue that the select into outfile statement might have no result when the column type is YEAR #22159
    • Fix the issue that the query result might be wrong when NULL is in the UNION subquery #26532
    • Fix the issue that the projection operator in execution might cause panic in some cases #26534

  • TiKV

    • Fix the issue that the duration calculation might panic on certain platforms #related-issue
    • Fix the wrong function that casts DOUBLE to DOUBLE #25200
    • Fix the issue that the panic log might be lost when using the async logger #8998
    • Fix the panic issue that occurs when building a snapshot twice if encryption is enabled #9786 #10407
    • Fix the wrong arguments type of the json_unquote() function in the coprocessor #10176
    • Fix the issues of suspicious warnings during shutdown and the non-deterministic response from Raftstore #10353 #10307
    • Fix the issue of backup threads leak #10287
    • Fix the issue that Region split might panic and corrupt the metadata if the split process is too slow and Region merge is on-going #8456 #8783
    • Fix the issue that the Region heartbeats prevent TiKV from splitting large Regions in some situations #10111
    • Fix the wrong statistics caused by the format inconsistency of CM Sketch between TiKV and TiDB #25638
    • Fix the wrong statistics of the apply wait duration metric #9893
    • Fix the “Missing Blob” error after using delete_files_in_range in Titan #10232

  • PD

    • Fix a bug that the scheduler might reappear after executing the delete operation #2572
    • Fix the data race issue that might occur when the scheduler is started before the temporary configuration is loaded #3771
    • Fix a PD panic issue that might occur during the Region scattering operation #3761
    • Fix the issue that the priority of some operators is not set correctly #3703
    • Fix a PD panic issue that might occur when deleting the evict-leader scheduler from a non-existent store #3660
    • Fix the issue that the PD Leader re-election is slow when there are many stores #3697

  • TiDB Dashboard

    • Fix the issue that the Profiling UI cannot profile all TiDB instances #944
    • Fix the issue that the Statements UI does not display “Plan Count” #939
    • Fix the issue that the Slow Query UI might display the “unknown field” error after cluster upgrade #902

  • TiFlash

    • Fix the potential panic issue that occurs when compiling DAG requests
    • Fix the panic issue that occurs when the read load is heavy
    • Fix the issue that TiFlash keeps restarting because of the split failure in column storage
    • Fix a potential bug that TiFlash cannot delete the delta data
    • Fix the incorrect results that occur when cloning the shared delta index concurrently
    • Fix a bug that TiFlash fails to restart in the case of incomplete data
    • Fix the issue that the old dm files cannot be removed automatically
    • Fix the panic issue that occurs when executing the SUBSTRING function with specific arguments
    • Fix the issue of incorrect results when casting the INTEGER type to the TIME type

  • Tools

    • Backup & Restore (BR)

      • Fix the issue that the data restore from the mysql schema might fail #1142

    • TiDB Lightning

      • Fix the issue that TiDB Lightning fails to parse the DECIMAL type data in Parquet files #1276
      • Fix the EOF error reported when TiDB Lightning splits the imported large CSV files #1133
      • Fix a bug that an excessively large base value is generated when TiDB Lightning imports tables with the auto_increment column of the FLOAT or DOUBLE type #1185
      • Fix the issue of TiDB Lightning panic that occurs when generating KV data larger than 4 GB #1128

    • Dumpling

      • When using Dumpling to export data to the S3 storage, the s3:ListBucket permission is no longer required on the entire bucket. The permission is required only on the data source prefix. #898

    • TiCDC

      • Fix the issue of extra partition dispatching after adding new table partitions #2205
      • Fix the panic issue that occurs when TiCDC fails to read /proc/meminfo #2023
      • Reduce TiCDC’s runtime memory consumption #2011 #1957
      • Fix a bug that some MySQL connection might leak after MySQL sink meets the error and pauses #1945
      • Fix the issue that TiCDC changefeed cannot be created when start TS is less than current TS minus GC TTL #1839
      • Reduce memory malloc in sort heap to avoid too much CPU overhead #1853
      • Fix a bug that the replication task might stop when moving a table #1827