TiDB 6.5.9 Release Notes

Release date: April 12, 2024

TiDB version: 6.5.9

Quick access: Quick start | Production deployment

Compatibility changes

• Add a TiKV configuration item track-and-verify-wals-in-manifest for RocksDB, which helps you investigate possible corruption of Write Ahead Log (WAL) #16549 @v01dstar
• DR Auto-Sync supports configuring wait-recover-timeout, which enables you to control the waiting time for switching back to the sync-recover status after the network recovers #6295 @disksing

Improvements

• TiDB

  • When force-init-stats is set to true, TiDB waits for statistics initialization to finish before providing services during TiDB startup. This setting no longer blocks the startup of HTTP servers, which enables users to continue monitoring #50854 @hawkingrei
  • Optimize the issue that the ANALYZE statement blocks the metadata lock #47475 @wjhuang2016

• TiKV

  • Remove unnecessary async blocks to reduce memory usage #16540 @overvenus
  • Avoid performing IO operations on snapshot files in raftstore threads to improve TiKV stability #16564 @Connor1996
  • Add slow logs for peer and store messages #16600 @Connor1996

• Tools

  • Backup & Restore (BR)

    • Optimize the Recovery Point Objective (RPO) for log backup during rolling restarts. Now, the checkpoint lag of log backup tasks will be smaller during rolling restarts #15410 @YuJuncen
    • Enhance the tolerance of log backup to merge operations. When encountering a reasonably long merge operation, log backup tasks are less likely to enter the error state #16554 @YuJuncen
    • Support automatically abandoning log backup tasks when encountering a large checkpoint lag, to avoid prolonged blocking GC and potential cluster issues #50803 @RidRisR
    • Alleviate the issue that the latency of the PITR log backup progress increases when Region leadership migration occurs #13638 @YuJuncen
    • Improve the speed of merging SST files during data restore by using a more efficient algorithm #50613 @Leavrth
    • Support ingesting SST files in batch during data restore #16267 @3pointer
    • Remove an outdated compatibility check when using Google Cloud Storage (GCS) as the external storage #50533 @lance6716
    • Print the information of the slowest Region that affects global checkpoint advancement in logs and metrics during log backups #51046 @YuJuncen
    • Refactor the BR exception handling mechanism to increase tolerance for unknown errors #47656 @3pointer

Bug fixes

• TiDB

  • Fix the issue that after a large number of tables are created, the newly created tables might lack the stats_meta information, causing subsequent query estimation to fail to get accurate row count information #36004 @xuyifangreeneyes
  • Fix the issue that dropped tables are still counted by the Grafana Stats Healthy Distribution panel #39349 @xuyifangreeneyes
  • Fix the issue that TiDB does not handle the WHERE <column_name> filtering condition in a SQL statement when the query of that statement involves the MemTableScan operator #40937 @zhongzc
  • Fix the issue that query results might be incorrect when the HAVING clause in a subquery contains correlated columns #51107 @hawkingrei
  • Fix the issue that query results might be incorrect when you use Common Table Expressions (CTE) to access partitioned tables with missing statistics #51873 @qw4990
  • Fix the issue that query execution using MPP might lead to incorrect query results when a SQL statement contains JOIN and the SELECT list in the statement contains only constants #50358 @yibin87
  • Fix the issue that the AUTO_INCREMENT attribute causes non-consecutive IDs due to unnecessary transaction conflicts when assigning auto-increment IDs #50819 @tiancaiamao
  • Fix the issue that the monitoring metric tidb_statistics_auto_analyze_total on Grafana is not displayed as an integer #51051 @hawkingrei
  • Fix the issue that errors might be returned during the concurrent merging of global statistics for partitioned tables #48713 @hawkingrei
  • Fix the issue that getting the default value of a column returns an error if the column default value is dropped #50043 #51324 @crazycs520
  • Fix the issue that the INSERT ignore statement cannot fill in default values when the column is write-only #40192 @YangKeao
  • Fix the issue that TiDB crashes when shuffleExec quits unexpectedly #48230 @wshwsh12
  • Fix the goroutine leak issue that might occur when the HashJoin operator fails to spill to disk #50841 @wshwsh12
  • Fix the issue that renaming tables does not take effect when committing multiple statements in a transaction #39664 @tiancaiamao
  • Fix the issue that the query result is incorrect when the IN() predicate contains NULL #51560 @winoros
  • Fix the issue that querying JSON of BINARY type might cause an error in some cases #51547 @YangKeao
  • Fix the issue that parallel Apply might generate incorrect results when the table has a clustered index #51372 @guo-shaoge
  • Fix the issue that the init-stats process might cause TiDB to panic and the load stats process to quit #51581 @hawkingrei
  • Fix the issue that the tidb_merge_partition_stats_concurrency variable does not take effect when auto analyze is processing partitioned tables #47594 @hawkingrei
  • Fix the issue that after the time window for automatic statistics updates is configured, statistics might still be updated outside that time window #49552 @hawkingrei
  • Fix the issue that the approx_percentile function might cause TiDB panic #40463 @xzhangxian1008
  • Fix the issue that BIT type columns might cause query errors due to decode failures when they are involved in calculations of some functions #49566 #50850 #50855 @jiyfhust
  • Fix the goroutine leak issue that occurs when the memory usage of CTE queries exceed limits #50337 @guo-shaoge
  • Fix the issue that TiDB does not listen to the corresponding port when force-init-stats is configured #51473 @hawkingrei
  • Fix the issue that ALTER TABLE ... COMPACT TIFLASH REPLICA might incorrectly end when the primary key type is VARCHAR #51810 @breezewish
  • Fix the issue that the tidb_gogc_tuner_threshold system variable is not adjusted accordingly after the tidb_server_memory_limit variable is modified #48180 @hawkingrei
  • Fix the Can't find column ... error that might occur when aggregate functions are used for group calculations #50926 @qw4990
  • Fix the issue that the REVERSE function reports an error when processing columns of the BIT type #50850 #50855 @jiyfhust
  • Fix the issue that INSERT IGNORE reports an error when inserting data in bulk into a table undergoing a DDL operation #50993 @YangKeao
  • Fix the issue that the TiDB server adds a label via the HTTP interface and returns success, but does not take effect #51427 @you06
  • Fix the issue that the type returned by the IFNULL function is inconsistent with MySQL #51765 @YangKeao
  • Fix the issue that the TiDB server is marked as health before the initialization is complete #51596 @shenqidebaozi
  • Fix the issue that querying the TIDB_HOT_REGIONS table might incorrectly return INFORMATION_SCHEMA tables #50810 @Defined2014
  • Fix the issue that EXCHANGE PARTITION incorrectly processes foreign keys #51807 @YangKeao
  • Fix the issue that executing queries containing CTEs causes TiDB to panic #41688 @srstack

• TiKV

  • Fix the issue that after the process of destroying Peer is interrupted by applying snapshots, it does not resume even after applying snapshots is complete #16561 @tonyxuqqi
  • Fix the issue that inactive Write Ahead Logs (WALs) in RocksDB might corrupt data #16705 @Connor1996
  • Fix the issue that TiKV converts the time zone incorrectly for Brazil and Egypt #16220 @overvenus
  • Fix the issue that the monitoring metric tikv_unified_read_pool_thread_count has no data in some cases #16629 @YuJuncen
  • Fix the issue that JSON integers greater than the maximum INT64 value but less than the maximum UINT64 value are parsed as FLOAT64 by TiKV, resulting in inconsistency with TiDB #16512 @YangKeao
  • Fix the issue that during the execution of an optimistic transaction, if other transactions initiate the resolving lock operation on it, there is a small chance that the atomicity of the transaction might be broken if the transaction’s primary key has data that was previously committed in Async Commit or 1PC mode #16620 @MyonKeminta

• PD

  • Fix the issue that the scaling progress is not correctly displayed #7726 @CabinfeverB
  • Fix the issue that data race occurs when the MergeLabels function is called #7535 @lhy1024
  • Fix the issue that the PD monitoring item learner-peer-count does not synchronize the old value after a leader switch #7728 @CabinfeverB
  • Fix the issue that the query result of SHOW CONFIG includes the deprecated configuration item trace-region-flow #7917 @rleungx

• TiFlash

  • Fix the issue that TiFlash might panic due to unstable network connections with PD during replica migration #8323 @JaySon-Huang
  • Fix the issue that TiFlash might crash due to data race in case of remote reads #8685 @solotzg
  • Fix the issue that the ENUM column might cause TiFlash to crash during chunk encoding #8674 @yibin87
  • Fix the issue that TiFlash might panic when you insert data to columns with invalid default values in non-strict sql_mode #8803 @Lloyd-Pottiger
  • Fix the issue that if Region migration, split, or merge occurs after the precision of a TIME column is modified, queries might fail #8601 @JaySon-Huang

• Tools

  • Backup & Restore (BR)

    • Fix the issue that too many logs are printed when a full backup fails #51572 @Leavrth
    • Fix the issue that removing a log backup task after it is paused does not immediately restore the GC safepoint #52082 @3pointer
    • Fix the issue that BR could not back up the AUTO_RANDOM ID allocation progress in a union clustered index that contains an AUTO_RANDOM column #52255 @Leavrth
    • Fix the issue that stopping a log backup task causes TiDB to crash #50839 @YuJuncen
    • Fix the issue that TiKV panics when a full backup fails to find a peer in some extreme cases #16394 @Leavrth

  • TiCDC

    • Fix the issue that snapshot lost caused by GC is not reported in time when resuming a changefeed and the checkpoint-ts of the changefeed is smaller than the GC safepoint of TiDB #10463 @sdojjy
    • Fix the issue that data is written to a wrong CSV file due to wrong BarrierTS in scenarios where DDL statements are executed frequently #10668 @lidezhu
    • Fix the issue that the Syncpoint table might be incorrectly replicated #10576 @asddongmen
    • Fix the issue TiCDC panics when scheduling table replication tasks #10613 @CharlesCheung96
    • Fix the issue that data race in the KV client causes TiCDC to panic #10718 @asddongmen
    • Fix the issue that the file sequence number generated by the storage service might not increment correctly when using the storage sink #10352 @CharlesCheung96
    • Fix the issue that TiCDC cannot access Azure and GCS properly in storage sink scenarios #10592 @CharlesCheung96
    • Fix the issue that the old value part of open-protocol incorrectly outputs the default value according to the STRING type instead of its actual type #10803 @3AceShowHand
    • Fix the issue that a changefeed with eventual consistency enabled might fail when the object storage sink encounters a temporary failure #10710 @CharlesCheung96

  • TiDB Data Migration (DM)

    • Fix the issue that data is lost when the upstream primary key is of binary type #10672 @GMHDBJD

  • TiDB Lightning

    • Fix the issue that TiDB Lightning reports an error when encountering invalid symbolic link files during file scanning #49423 @lance6716