TiDB 7.5.1 Release Notes
Release date: February 29, 2024
TiDB version: 7.5.1
Quick access: Quick start | Production deployment
Compatibility changes
- Prohibit setting require_secure_transport to
ON
in Security Enhanced Mode (SEM) to prevent potential connectivity issues for users #47665 @tiancaiamao - To reduce the overhead of log printing, TiFlash changes the default value of
logger.level
from"debug"
to"info"
#8641 @JaySon-Huang - Introduce the TiKV configuration item gc.num-threads to set the number of GC threads when
enable-compaction-filter
isfalse
#16101 @tonyxuqqi - TiCDC Changefeed introduces the following new configuration items:
- compression: enables you to configure the compression behavior of redo log files #10176 @sdojjy
- sink.cloud-storage-config: enables you to set the automatic cleanup of historical data when replicating data to object storage #10109 @CharlesCheung96
- consistent.flush-concurrency: enables you to set the concurrency for uploading a single redo file #10226 @sdojjy
Improvements
TiDB
Use
tikv_client_read_timeout
during the DDL schema reload process to reduce the impact of Meta Region Leader read unavailability on the cluster #48124 @cfzjywxkEnhance observability related to resource control #49318 @glorv @bufferflies @nolouch
As more and more users use resource groups to isolate application workloads, Resource Control provides enhanced data based on resource groups. This helps you monitor resource group workloads and settings, ensuring that you can quickly identify and accurately diagnose problems, including:
- Slow Queries: add the resource group name, resource unit (RU) consumption, and time for waiting for resources.
- Statement Summary Tables: add the resource group name, RU consumption, and time for waiting for resources.
- In the system variable tidb_last_query_info, add a new entry
ru_consumption
to indicate the consumed RU by SQL statements. You can use this variable to get the resource consumption of the last statement in the session. - Add database metrics based on resource groups: QPS/TPS, execution time (P999/P99/P95), number of failures, and number of connections.
Modify the
CANCEL IMPORT JOB
statement to a synchronous statement #48736 @D3HunterSupport the FLASHBACK CLUSTER TO TSO syntax #48372 @BornChanger
Optimize the TiDB implementation when handling some type conversions and fix related issues #47945 #47864 #47829 #47816 @YangKeao @lcwangchao
When a non-binary collation is set and the query includes
LIKE
, the optimizer generates anIndexRangeScan
to improve the execution efficiency #48181 #49138 @time-and-fateEnhance the ability to convert
OUTER JOIN
toINNER JOIN
in specific scenarios #49616 @qw4990Support multiple accelerated
ADD INDEX
DDL tasks to be queued for execution, instead of falling back to normalADD INDEX
tasks #47758 @tangentaImprove the speed of adding indexes to empty tables #49682 @zimulala
TiKV
- Enhance the slow store detection algorithm by improving its sensitivity and reducing the false-positive rate, especially in intensive read and write load scenarios #15909 @LykxSassinator
TiFlash
- Improve the calculation method for Request Unit (RU) to make RU values more stable #8391 @guo-shaoge
- Reduce the impact of disk performance jitter on read latency #8583 @JaySon-Huang
- Reduce the impact of background GC tasks on read and write task latency #8650 @JaySon-Huang
Tools
Backup & Restore (BR)
- Improve the speed of merging SST files during data restore by using a more efficient algorithm #50613 @Leavrth
- Support creating databases in batch during data restore #50767 @Leavrth
- Support ingesting SST files in batch during data restore #16267 @3pointer
- Print the information of the slowest Region that affects global checkpoint advancement in logs and metrics during log backups #51046 @YuJuncen
- Improve the table creation performance of the
RESTORE
statement in scenarios with large datasets #48301 @Leavrth - BR can pause Region merging by setting the
merge-schedule-limit
configuration to0
#7148 @BornChanger - Refactor the BR exception handling mechanism to increase tolerance for unknown errors #47656 @3pointer
TiCDC
- Support searching TiCDC logs in the TiDB Dashboard #10263 @CharlesCheung96
- Support querying the downstream synchronization status of a changefeed, which helps you determine whether the upstream data changes received by TiCDC have been synchronized to the downstream system completely #10289 @hongyunyan
- Improve the performance of TiCDC replicating data to object storage by increasing parallelism #10098 @CharlesCheung96
TiDB Lightning
Bug fixes
TiDB
- Fix the issue that setting the system variable
tidb_service_scope
does not take effect #49245 @ywqzzy - Fix the issue that the communication protocol cannot handle packets larger than or equal to 16 MB when compression is enabled #47157 #47161 @dveeden
- Fix the issue that the
approx_percentile
function might cause TiDB panic #40463 @xzhangxian1008 - Fix the issue that TiDB might implicitly insert the
from_binary
function when the argument of a string function is aNULL
constant, causing some expressions unable to be pushed down to TiFlash #49526 @YangKeao - Fix the goroutine leak issue that might occur when the
HashJoin
operator fails to spill to disk #50841 @wshwsh12 - Fix the issue that
BIT
type columns might cause query errors due to decode failures when they are involved in calculations of some functions #49566 #50850 #50855 @jiyfhust - Fix the goroutine leak issue that occurs when the memory usage of CTE queries exceed limits #50337 @guo-shaoge
- Fix the issue that wrong results might be returned when TiFlash late materialization processes associated columns #49241 #51204 @Lloyd-Pottiger
- Fix the issue that the background job thread of TiDB might panic when TiDB records historical statistics #49076 @hawkingrei
- Fix the error that might occur when TiDB merges histograms of global statistics for partitioned tables #49023 @hawkingrei
- Fix the issue that the historical statistics of the
stats_meta
table are not updated after a partition is dropped #49334 @hi-rustin - Fix the issue of incorrect query results caused by multi-valued indexes mistakenly selected as the
Index Join
probe side #50382 @AilinKid - Fix the issue that the
USE_INDEX_MERGE
hint does not take effect on multi-valued indexes #50553 @AilinKid - Fix the issue that users might get errors when querying the
INFORMATION_SCHEMA.ANALYZE_STATUS
system table #48835 @hi-rustin - Fix the issue of wrong query results due to TiDB incorrectly eliminating constant values in
group by
#38756 @hi-rustin - Fix the issue that the
processed_rows
of theANALYZE
task on a table might exceed the total number of rows in that table #50632 @hawkingrei - Fix the issue that TiDB might panic when using the
EXECUTE
statement to executePREPARE STMT
after thetidb_enable_prepared_plan_cache
system variable is enabled and then disabled #49344 @qw4990 - Fix the
Column ... in from clause is ambiguous
error that might occur when a query usesNATURAL JOIN
#32044 @AilinKid - Fix the issue that using a multi-valued index to access an empty JSON array might return incorrect results #50125 @YangKeao
- Fix the
Can't find column ...
error that might occur when aggregate functions are used for group calculations #50926 @qw4990 - Fix the issue that the control of
SET_VAR
for variables of the string type might become invalid #50507 @qw4990 - Fix the issue that high CPU usage of TiDB occurs due to long-term memory pressure caused by
tidb_server_memory_limit
#48741 @XuHuaiyu - Fix the issue that the completion times of two DDL tasks with dependencies are incorrectly sequenced #49498 @tangenta
- Fix the issue that illegal optimizer hints might cause valid hints to be ineffective #49308 @hawkingrei
- Fix the issue that DDL statements with the
CHECK
constraint are stuck #47632 @jiyfhust - Fix the issue that the behavior of the
ENFORCED
option in theCHECK
constraint is inconsistent with MySQL 8.0 #47567 #47631 @jiyfhust - Fix the issue that CTE queries might report an error
type assertion for CTEStorageMap failed
during the retry process #46522 @tiancaiamao - Fix the issue that the
DELETE
andUPDATE
statements using index lookup might report an error whentidb_multi_statement_mode
mode is enabled #50012 @tangenta - Fix the issue that
UPDATE
orDELETE
statements containingWITH RECURSIVE
CTEs might produce incorrect results #48969 @winoros - Fix the issue that the optimizer incorrectly converts TiFlash selection path to the DUAL table in specific scenarios #49285 @AilinKid
- Fix the issue that the same query plan has different
PLAN_DIGEST
values in some cases #47634 @King-Dylan - Fix the issue that after the time window for automatic statistics updates is configured, statistics might still be updated outside that time window #49552 @hawkingrei
- Fix the issue that the query result is incorrect when an
ENUM
type column is used as the join key #48991 @winoros - Fix the issue that executing
UNIQUE
index lookup with anORDER BY
clause might cause an error #49920 @jackysp - Fix the issue that
LIMIT
in multi-level nestedUNION
queries might become ineffective #49874 @Defined2014 - Fix the issue that the result of
COUNT(INT)
calculated by MPP might be incorrect #48643 @AilinKid - Fix the issue that parsing invalid values of
ENUM
orSET
types would directly cause SQL statement errors #49487 @winoros - Fix the issue that TiDB panics and reports an error
invalid memory address or nil pointer dereference
#42739 @CbcWestwolf - Fix the issue that executing
UNION ALL
with the DUAL table as the first subnode might cause an error #48755 @winoros - Fix the issue that common hints do not take effect in
UNION ALL
statements #50068 @hawkingrei - Fix the issue that TiDB server might panic during graceful shutdown #36793 @bb7133
- Fix the issue that Daylight Saving Time is displayed incorrectly in some time zones #49586 @overvenus
- Fix the issue that static
CALIBRATE RESOURCE
relies on the Prometheus data #49174 @glorv - Fix the issue that hints cannot be used in
REPLACE INTO
statements #34325 @YangKeao - Fix the issue that executing queries containing the
GROUP_CONCAT(ORDER BY)
syntax might return errors #49986 @AilinKid - Fix the issue that TiDB server might consume a significant amount of resources when the enterprise plugin for audit logging is used #49273 @lcwangchao
- Fix the issue that using old interfaces might cause inconsistent metadata for tables #49751 @hawkingrei
- Fix the issue that disabling
tidb_enable_collect_execution_info
causes the coprocessor cache to panic #48212 @you06 - Fix the issue that executing
ALTER TABLE ... LAST PARTITION
fails when the partition column type isDATETIME
#48814 @crazycs520 - Fix the issue that the
COMMIT
orROLLBACK
operation executed throughCOM_STMT_EXECUTE
fails to terminate transactions that have timed out #49151 @zyguan - Fix the issue that histogram statistics might not be parsed into readable strings when the histogram boundary contains
NULL
#49823 @AilinKid - Fix the issue that queries containing common table expressions (CTEs) unexpectedly get stuck when the memory limit is exceeded #49096 @AilinKid
- Fix the issue that data is inconsistent under the TiDB Distributed eXecution Framework (DXF) when executing
ADD INDEX
after the DDL Owner is network isolated #49773 @tangenta - Fix the issue that the auto-increment ID allocation reports an error due to concurrent conflicts when using an auto-increment column with
AUTO_ID_CACHE=1
#50519 @tiancaiamao - Fix the issue that TiDB might panic when a query contains the Apply operator and the
fatal error: concurrent map writes
error occurs #50347 @SeaRise - Fix the TiDB node panic issue that occurs when DDL
jobID
is restored to 0 #46296 @jiyfhust - Fix the issue that query results are incorrect due to
STREAM_AGG()
incorrectly handling CI #49902 @wshwsh12 - Mitigate the issue that TiDB nodes might encounter OOM errors when dealing with a large number of tables or partitions #50077 @zimulala
- Fix the issue that the
LEADING
hint does not take effect inUNION ALL
statements #50067 @hawkingrei - Fix the issue that
LIMIT
andOPRDERBY
might be invalid in nestedUNION
queries #49377 @AilinKid - Fix the issue that a query containing the IndexHashJoin operator gets stuck when memory exceeds
tidb_mem_quota_query
#49033 @XuHuaiyu - Fix the issue that TiDB returns wrong query results when processing
ENUM
orSET
types by constant propagation #49440 @winoros - Fix the issue that executing
SELECT INTO OUTFILE
using thePREPARE
method incorrectly returns a success message instead of an error #49166 @qw4990 - Fix the issue that enforced sorting might become ineffective when a query uses optimizer hints (such as
STREAM_AGG()
) that enforce sorting and its execution plan containsIndexMerge
#49605 @AilinKid - Fix the issue that tables with
AUTO_ID_CACHE=1
might lead to gRPC client leaks when there are a large number of tables #48869 @tiancaiamao - Fix the issue that in non-strict mode (
sql_mode = ''
), truncation during executingINSERT
still reports an error #49369 @tiancaiamao - Fix the issue that using the
_
wildcard inLIKE
when the data contains trailing spaces can result in incorrect query results #48983 @time-and-fate - Fix the issue that executing
ADMIN CHECK
after updating thetidb_mem_quota_query
system variable returnsERROR 8175
#49258 @tangenta - Fix the issue of excessive statistical error in constructing statistics caused by Golang’s implicit conversion algorithm #49801 @qw4990
- Fix the issue that queries containing CTEs report
runtime error: index out of range [32] with length 32
whentidb_max_chunk_size
is set to a small value #48808 @guo-shaoge
- Fix the issue that setting the system variable
TiKV
- Fix the issue that enabling
tidb_enable_row_level_checksum
might cause TiKV to panic #16371 @cfzjywxk - Fix the issue that TiKV might panic when gRPC threads are checking
is_shutdown
#16236 @pingyu - Fix the issue that TiKV converts the time zone incorrectly for Brazil and Egypt #16220 @overvenus
- Fix the issue that
blob-run-mode
in Titan cannot be updated online #15978 @tonyxuqqi - Fix the issue that TiDB and TiKV might produce inconsistent results when processing
DECIMAL
arithmetic multiplication truncation #16268 @solotzg - Fix the issue that Flashback might get stuck when encountering
notLeader
orregionNotFound
#15712 @HuSharp - Fix the issue that damaged SST files might be spread to other TiKV nodes #15986 @Connor1996
- Fix the issue that if TiKV runs extremely slowly, it might panic after Region merge #16111 @overvenus
- Fix the issue that the joint state of DR Auto-Sync might time out when scaling out #15817 @Connor1996
- Fix the issue that Resolved TS might be blocked for two hours #11847 #15520 #39130 @overvenus
- Fix the issue that
cast_duration_as_time
might return incorrect results #16211 @gengliqi - Fix the issue that TiKV hangs in corner cases (for example, when disk I/O operations are blocked), which affects availability #16368 @LykxSassinator
- Fix the issue that enabling
PD
- Fix the issue that querying resource groups in batch might cause PD to panic #7206 @nolouch
- Fix the issue that PD cannot read resource limitations when it is started with
systemd
#7628 @bufferflies - Fix the issue that continuous jitter in PD disk latency might cause PD to fail to select a new leader #7251 @HuSharp
- Fix the issue that a network partition in PD might cause scheduling not to be started immediately #7016 @HuSharp
- Fix the issue that the PD monitoring item
learner-peer-count
does not synchronize the old value after a leader switch #7728 @CabinfeverB - Fix the issue that when PD leader is transferred and there is a network partition between the new leader and the PD client, the PD client fails to update the information of the leader #7416 @CabinfeverB
- Fix some security issues by upgrading the version of Gin Web Framework from v1.8.1 to v1.9.1 #7438 @niubell
- Fix the issue that the orphan peer is deleted when the number of replicas does not meet the requirements #7584 @bufferflies
- Fix the issue that querying a Region without a leader using
pd-ctl
might cause PD to panic #7630 @rleungx
TiFlash
- Fix the issue that TiFlash might panic due to unstable network connections with PD during replica migration #8323 @JaySon-Huang
- Fix the issue that removing and then re-adding TiFlash replicas might lead to data corruption in TiFlash #8695 @JaySon-Huang
- Fix a potential issue that
FLASHBACK TABLE
orRECOVER TABLE
might fail to recover data of some TiFlash replicas ifDROP TABLE
is executed immediately after data insertion #8395 @JaySon-Huang - Fix incorrect display of maximum percentile time for some panels in Grafana #8076 @JaySon-Huang
- Fix the issue that TiFlash might crash during remote reads #8685 @guo-shaoge
- Fix the issue that TiFlash incorrectly handles
ENUM
when theENUM
value is 0 #8311 @solotzg - Fix the issue that short queries executed successfully print excessive info logs #8592 @windtalker
- Fix the issue that the memory usage increases significantly due to slow queries #8564 @JinheLin
- Fix the issue that the
lowerUTF8
andupperUTF8
functions do not allow characters in different cases to occupy different bytes #8484 @gengliqi - Fix the potential OOM issue that might occur when scanning multiple partitioned tables during stream read #8505 @gengliqi
- Fix the issue of memory leak when TiFlash encounters memory limitation during query #8447 @JinheLin
- Fix the TiFlash panic issue when TiFlash encounters conflicts during concurrent DDL execution #8578 @JaySon-Huang
- Fix the issue that TiFlash panics after executing
ALTER TABLE ... MODIFY COLUMN ... NOT NULL
, which changes nullable columns to non-nullable #8419 @JaySon-Huang - Fix the issue that query results are incorrect when querying with filtering conditions like
ColumnRef in (Literal, Func...)
#8631 @Lloyd-Pottiger - Fix the issue that data of TiFlash replicas would still be garbage collected after executing
FLASHBACK DATABASE
#8450 @JaySon-Huang - Fix the issue that TiFlash might not be able to select the GC owner of object storage data under the disaggregated storage and compute architecture #8519 @JaySon-Huang
- Fix the random invalid memory access issue that might occur with
GREATEST
orLEAST
functions containing constant string parameters #8604 @windtalker - Fix the issue that TiFlash replica data might be accidentally deleted after performing point-in-time recovery (PITR) or executing
FLASHBACK CLUSTER TO
, which might result in data anomalies #8777 @JaySon-Huang - Fix the issue that TiFlash Anti Semi Join might return incorrect results when the join includes non-equivalent conditions #8791 @windtalker
Tools
Backup & Restore (BR)
- Fix the issue that data restore is slowed down due to absence of a leader on a TiKV node #50566 @Leavrth
- Fix the issue that full restore still requires the target cluster to be empty after the
--filter
option is specified #51009 @3pointer - Fix the issue that when resuming from a checkpoint after data restore fails, an error
the target cluster is not fresh
occurs #50232 @Leavrth - Fix the issue that stopping a log backup task causes TiDB to crash #50839 @YuJuncen
- Fix the issue that the
Unsupported collation
error is reported when you restore data from backups of an old version #49466 @3pointer - Fix the issue that the log backup task can start but does not work properly if failing to connect to PD during task initialization #16056 @YuJuncen
- Fix the issue that BR generates incorrect URIs for external storage files #48452 @3AceShowHand
- Fix the issue that log backup gets stuck after changing the TiKV IP address on the same node #50445 @3pointer
- Fix the issue that BR cannot retry when encountering an error while reading file content from S3 #49942 @Leavrth
TiCDC
- Fix the issue that the sink module fails to restart correctly after encountering an error when Syncpoint is enabled (
enable-sync-point = true
) #10091 @hicqu - Fix the issue that the file sequence number generated by the storage service might not increment correctly when using the storage sink #10352 @CharlesCheung96
- Fix the issue that the Syncpoint table might be incorrectly replicated #10576 @asddongmen
- Fix the issue that OAuth2.0, TLS, and mTLS cannot be enabled properly when using Apache Pulsar as the downstream #10602 @asddongmen
- Fix the issue that TiCDC returns the
ErrChangeFeedAlreadyExists
error when concurrently creating multiple changefeeds #10430 @CharlesCheung96 - Fix the issue that the changefeed
resolved ts
does not advance in extreme cases #10157 @sdojjy - Fix the issue that TiCDC mistakenly closes the connection with TiKV in certain special scenarios #10239 @hicqu
- Fix the issue that the TiCDC server might panic when replicating data to an object storage service #10137 @sdojjy
- Fix the issue that the changefeed reports an error after
TRUNCATE PARTITION
is executed on the upstream table #10522 @sdojjy - Fix the issue that after filtering out
add table partition
events is configured inignore-event
, TiCDC does not replicate other types of DML changes for related partitions to the downstream #10524 @CharlesCheung96 - Fix the potential data race issue during
kv-client
initialization #10095 @3AceShowHand
- Fix the issue that the sink module fails to restart correctly after encountering an error when Syncpoint is enabled (
TiDB Data Migration (DM)