TiDB 6.1.1 Release Notes

Release date: September 1, 2022

TiDB version: 6.1.1

Quick access: Quick start | Production deployment

Compatibility changes

• TiDB

  • Make the SHOW DATABASES LIKE … statement case-insensitive #34766 @e1ijah1
  • Change the default value of tidb_enable_outer_join_reorder from 1 to 0, which disables Join Reorder’s support for Outer Join by default.

• Diagnosis

  • Disable the Continuous Profiling feature by default, which avoids the possible TiFlash crash issue that occurs when this feature is enabled. For details, see #5687 @mornyx

Other changes

• Add the following contents in the TiDB-community-toolkit binary package. For details, see TiDB Installation Packages.

  • server-{version}-linux-amd64.tar.gz
  • grafana-{version}-linux-amd64.tar.gz
  • alertmanager-{version}-linux-amd64.tar.gz
  • prometheus-{version}-linux-amd64.tar.gz
  • blackbox_exporter-{version}-linux-amd64.tar.gz
  • node_exporter-{version}-linux-amd64.tar.gz
• Introduce multi-level support for different quality standards on the combination of operating systems and CPU architectures. See OS and platform requirements.

Improvements

• TiDB

  • Add a new optimizer SEMI_JOIN_REWRITE to improve the performance of EXISTS queries #35323 @winoros

• TiKV

  • Support compressing the metrics response using gzip to reduce the HTTP body size #12355 @winoros
  • Support reducing the amount of data returned for each request by filtering out some metrics using the server.simplify-metrics configuration item #12355 @glorv
  • Support dynamically modifying the number of sub-compaction operations performed concurrently in RocksDB (rocksdb.max-sub-compactions) #13145 @ethercflow

• PD

  • Improve the scheduling speed of Balance Region in specific stages #4990 @bufferflies

• Tools

  • TiDB Lightning

    • Add a retry mechanism on errors such as stale command to improve import success rate #36877 @D3Hunter

  • TiDB Data Migration (DM)

    • Users can manually set the amount of concurrency for lightning loader #5505 @buchuitoudegou

  • TiCDC

    • Add a sink uri parameter transaction-atomicity to support splitting the large transaction in a changefeed. This can greatly reduce the latency and memory consumption of large transactions #5231 @CharlesCheung96
    • Reduce performance overhead caused by runtime context switching in multi-Region scenarios #5610 @hicqu
    • Enhance the MySQL sink to turn off the safe mode automatically #5611 @overvenus

Bug fixes

• TiDB

  • Fix the issue that INL_HASH_JOIN might hang when used with LIMIT #35638 @guo-shaoge
  • Fix the issue that TiDB might panic when executing the UPDATE statement #32311 @Yisaer
  • Fix a bug that TiDB might send coprocessor requests when executing the SHOW COLUMNS statement #36496 @tangenta
  • Fix a bug that TiDB might return the invalid memory address or nil pointer dereference error when executing the SHOW WARNINGS statement #31569 @zyguan
  • Fix a bug that in the static partition prune mode, SQL statements with an aggregate condition might return wrong result when the table is empty #35295 @tiancaiamao
  • Fix the issue that the Join Reorder operation will mistakenly push down its Outer Join condition #37238 @winoros
  • Fix the issue that CTE-schema hash code is cloned mistakenly, which causes the Can't find column ... in schema ... error when CTE is referenced more than once #35404 @AilinKid
  • Fix the issue that the wrong join reorder in some right outer join scenarios causes wrong query result #36912 @winoros
  • Fix the issue of incorrectly inferred null flag of the TiFlash firstrow aggregate function in the EqualAll case #34584 @fixdb
  • Fix the issue that Plan Cache does not work when a binding is created with the IGNORE_PLAN_CACHE hint #34596 @fzzf678
  • Fix the issu that an EXCHANGE operator is missing between the hash-partition window and the single-partition window #35990 @LittleFall
  • Fix the issue that partitioned tables cannot fully use indexes to scan data in some cases #33966 @mjonss
  • Fix the issue of wrong query result when a wrong default value is set for partial aggregation after the aggregation is pushed down #35295 @tiancaiamao
  • Fix the issue that querying partitioned tables might get the index-out-of-range error in some cases #35181 @mjonss
  • Fix the issue that a partition is incorrectly pruned if a partition key is used in the query condition and the collate is different from the one in the query partition table #32749 @mjonss
  • Fix the issue that when TiDB Binlog is enabled, executing the ALTER SEQUENCE statement might cause a wrong metadata version and cause Drainer to exit #36276 @AilinKid
  • Fix the issue of incorrect TiDB status that might appear on startup in some extreme cases #36791 @xhebox
  • Fix the potential UnknownPlanID issue that occurs when querying the execution plans for partitioned tables in TiDB Dashboard #35153 @time-and-fate
  • Fix the issue that the column list does not work in the LOAD DATA statement #35198 @SpadeA-Tang
  • Fix the issue of the data and columnID count not match error that occurs when inserting duplicated values with TiDB Binlog enabled #33608 @zyguan
  • Remove the limitation of tidb_gc_life_time #35392 @TonsnakeLin
  • Fix the LOAD DATA statement dead loop when an empty filed terminator is used #33298 @zyguan
  • Avoid sending requests to unhealthy TiKV nodes to improve availability #34906 @sticnarf

• TiKV

  • Fix a bug that Regions might be overlapped if Raftstore is busy #13160 @5kbpers
  • Fix the issue that PD does not reconnect to TiKV after the Region heartbeat is interrupted #12934 @bufferflies
  • Fix the issue that TiKV panics when performing type conversion for an empty string #12673 @wshwsh12
  • Fix the issue of inconsistent Region size configuration between TiKV and PD #12518 @5kbpers
  • Fix the issue that encryption keys are not cleaned up when Raft Engine is enabled #12890 @tabokie
  • Fix the panic issue that might occur when a peer is being split and destroyed at the same time #12825 @BusyJay
  • Fix the panic issue that might occur when the source peer catches up logs by snapshot in the Region merge process #12663 @BusyJay
  • Fix the issue of frequent PD client reconnection that occurs when the PD client meets an error #12345 @Connor1996
  • Fix potential panic when parallel recovery is enabled for Raft Engine #13123 @tabokie
  • Fix the issue that the Commit Log Duration of a new Region is too high, which causes QPS to drop #13077 @Connor1996
  • Fix rare panics when Raft Engine is enabled #12698 @tabokie
  • Avoid redundant log warnings when proc filesystem (procfs) cannot be found #13116 @tabokie
  • Fix the wrong expression of Unified Read Pool CPU in dashboard #13086 @glorv
  • Fix the issue that when a Region is large, the default region-split-check-diff might be larger than the bucket size #12598 @tonyxuqqi
  • Fix the issue that TiKV might panic when Apply Snapshot is aborted and Raft Engine is enabled #12470 @tabokie
  • Fix the issue that the PD client might cause deadlocks #13191 @bufferflies #12933 @BurtonQin

• PD

  • Fix the issue that the online progress is inaccurate when label configurations of cluster nodes are invalid #5234 @rleungx
  • Fix PD panics caused by the issue that gRPC handles errors inappropriately when enable-forwarding is enabled #5373 @bufferflies
  • Fix the issue that /regions/replicated might return a wrong status #5095 @rleungx

• TiFlash

  • Fix the issue that TiFlash crashes after dropping a column of a table with clustered indexes in some situations #5154 @hongyunyan
  • Fix the issue that the format function might return a Data truncated error #4891 @xzhangxian1008
  • Fix the issue that some obsolete data might persist in storage and cannot be deleted #5659 @lidezhu
  • Fix unnecessary CPU usage in some edge cases #5409 @breezewish
  • Fix a bug that TiFlash cannot work in a cluster using IPv6 #5247 @solotzg
  • Fix a bug that TiFlash might crash due to an error in parallel aggregation #5356 @gengliqi
  • Fix a bug that thread resources might leak in case of MinTSOScheduler query errors #5556 @windtalker

• Tools

  • TiDB Lightning

    • Fix the issue that TiDB Lightning fails to connect to TiDB when TiDB uses an IPv6 host #35880 @D3Hunter
    • Fix the read index not ready error by adding a retry mechanism #36566 @D3Hunter
    • Fix the issue that sensitive information in logs is printed in server mode #36374 @lichunzhu
    • Fix the issue that TiDB Lightning does not support columns starting with slash, number, or non-ascii characters in Parquet files #36980 @D3Hunter
    • Fix the issue that de-duplication might cause TiDB Lightning to panic in extreme cases #34163 @ForwardStar

  • TiDB Data Migration (DM)

    • Fix the issue that the txn-entry-size-limit configuration item does not take effect in DM #6161 @ForwardStar
    • Fix the issue that the check-task command cannot handle special characters #5895 @Ehco1996
    • Fix the issue of possible data race in query-status #4811 @lyzx2001
    • Fix the different output format for the operate-schema command #5688 @ForwardStar
    • Fix goroutine leak when relay meets an error #6193 @lance6716
    • Fix the issue that DM Worker might get stuck when getting DB Conn #3733 @lance6716
    • Fix the issue that DM fails to start when TiDB uses an IPv6 host #6249 @D3Hunter

  • TiCDC

    • Fix the wrong maximum compatible version number #6039 @hi-rustin
    • Fix a bug that may cause the cdc server to panic when it receives an HTTP request before it fully starts #5639 @asddongmen
    • Fix the ddl sink panic issue when the changefeed sync-point is enabled #4934 @asddongmen
    • Fix the issue that a changefeed is stuck in some scenarios when sync-point is enabled #6827 @hicqu
    • Fix a bug that changefeed API does not work properly after the cdc server restarts #5837 @asddongmen
    • Fix the data race issue in the black hole sink #6206 @asddongmen
    • Fix the TiCDC panic issue when you set enable-old-value = false #6198 @hi-rustin
    • Fix the data consistency issue when the redo log feature is enabled #6189 #6368 #6277 #6456 #6695 #6764 #6859 @asddongmen
    • Fix poor redo log performance by writing redo events asynchronously #6011 @CharlesCheung96
    • Fix the issue that the MySQL sink cannot connect to IPv6 addresses #6135 @hi-rustin

  • Backup & Restore (BR)

    • Fix a bug that BR reports ErrRestoreTableIDMismatch in RawKV mode #35279 @3pointer
    • Adjust the backup data directory structure to fix backup failure caused by S3 rate limiting in large cluster backup #30087 @MoCuishle28
    • Fix incorrect backup time in the summary log #35553 @ixuh12

  • Dumpling

    • Fix the issue that GetDSN does not support IPv6 #36112 @D3Hunter

  • TiDB Binlog

    • Fix a bug that Drainer cannot send requests correctly to Pump when compressor is set to gzip #1152 @lichunzhu