TiDB 4.0.16 Release Notes

TiDB 4.0.16 Release Notes

Release date: December 17, 2021

TiDB version: 4.0.16

Compatibility changes

TiKV
- Before v4.0.16, when TiDB converts an illegal UTF-8 string to a Real type, an error is reported directly. Starting from v4.0.16, TiDB processes the conversion according to the legal UTF-8 prefix in the string #11466
Tools
- TiCDC
  - Change the default value of Kafka Sink max-message-bytes to 1 MB to prevent TiCDC from sending too large messages to Kafka clusters #2962
  - Change the default value of Kafka Sink partition-num to 3 so that TiCDC distributes messages across Kafka partitions more evenly #3337

Improvements

TiDB
- Upgrade the Grafana version from 7.5.7 to 7.5.11
TiKV
- Reduce disk space consumption by adopting the zstd algorithm to compress SST files when restoring data using Backup & Restore or importing data using Local-backend of TiDB Lightning #11469
Tools
- Backup & Restore (BR)
  - Improve the robustness of restoring #27421
- TiCDC
  - Add a tick frequency limit to EtcdWorker to prevent frequent etcd writes from affecting PD services #3112
  - Optimize rate limiting control on TiKV reloads to reduce gPRC congestion during changefeed initialization #3110

Bug fixes

TiDB
- Fix the query panic caused by overflow in the statistics module when converting a range to points for cost estimation #23625
- Fix wrong results of the control functions (such as IF and CASE WHEN) when using the ENUM type data as parameters of such functions #23114
- Fix the issue that the GREATEST function returns inconsistent results due to different values of tidb_enable_vectorized_expression (on or off) #29434
- Fix the panic when applying index join on prefix indexes in some cases #24547
- Fix the issue that planner might cache invalid plans for join in some cases #28087
- Fix a bug that TiDB cannot insert null into a non-null column when sql_mode is empty #11648
- Fix the wrong result type of the GREATEST and LEAST functions #29019
- Fix the privilege check fail error when performing the grant and revoke operations to grant and revoke global level privileges #29675
- Fix the panic when using the CASE WHEN function on the ENUM data type #29357
- Fix wrong results of the microsecond function in vectorized expressions #29244
- Fix wrong results of the hour function in vectorized expression #28643
- Fix the issue that optimistic transaction conflicts might cause transactions to block each other #11148
- Fix the issue of incomplete log information from the auto analyze result #29188
- Fix the issue that using an invalid default date does not report an error when the SQL_MODE is ‘NO_ZERO_IN_DATE’ #26766
- Fix the issue that the Coprocessor Cache panel in Grafana does not display metrics. Now, Grafana displays the number of hits/miss/evict #26338
- Fix the issue that concurrently truncating the same partition causes DDL statements to stuck #26229
- Fix the issue that the length information is wrong when converting Decimal to String #29417
- Fix the issue of an extra column in the query result when NATURAL JOIN is used to join multiple tables #29481
- Fix the issue that TopN is wrongly pushed down to indexPlan when IndexScan uses a prefix index #29711
- Fix the issue that retrying transactions with the auto-increment columns of DOUBLE type causes data corruption #29892
TiKV
- Fix a panic issue that occurs when Region merge, ConfChange, and Snapshot happen at the same time in extreme conditions #11475
- Fix the issue of negative sign when the decimal divide result is zero #29586
- Fix the issue that the average latency of the by-instance gRPC requests is inaccurate in TiKV metrics #11299
- Fix the issue of TiCDC panic that occurs when the downstream database is missing #11123
- Fix the issue that the Raft connection is broken when the channel is full #11047
- Fix the issue that TiDB cannot correctly identify whether the Int64 types in Max/Min functions are a signed integer or not, which causes the wrong calculation result of Max/Min #10158
- Fix the issue that CDC adds scan retries frequently due to the Congest error #11082
PD
- Fix a panic issue that occurs after the TiKV node is removed #4344
- Fix slow leader election caused by stucked region syncer #3936
- Support that the evict leader scheduler can schedule regions with unhealthy peers #4093
TiFlash
- Fix the issue that TiFlash fails to start up on some platforms due to the absence of library nsl
Tools
- TiDB Binlog
  - Fix the bug that Drainer exits when transporting a transaction greater than 1 GB #28659
- TiCDC
  - Fix the negative value error in the changefeed checkpoint lag #3010
  - Fix OOM in container environments #1798
  - Fix the TiCDC replication interruption issue when multiple TiKVs crash or during a forced restart #3288
  - Fix the memory leak issue after processing DDLs #3174
  - Fix the issue that changefeed does not fail fast enough when the ErrGCTTLExceeded error occurs #3111
  - Fix the issue that TiCDC replication task might terminate when the upstream TiDB instance unexpectedly exits #3061
  - Fix the issue that TiCDC process might panic when TiKV sends duplicate requests to the same Region #2386
  - Fix the issue that the volume of Kafka messages generated by TiCDC is not constrained by max-message-size #2962
  - Fix the issue that tikv_cdc_min_resolved_ts_no_change_for_1m keeps alerting when there is no changefeed #11017
  - Fix the issue that TiCDC sync task might pause when an error occurs during writing a Kafka message #2978
  - Fix the issue that some partitioned tables without valid indexes might be ignored when force-replicate is enabled #2834
  - Fix the memory leak issue when creating a new changefeed #2389
  - Fix the issue that might cause inconsistent data due to Sink components advancing resolved ts early #3503
  - Fix the issue that scanning stock data might fail due to TiKV performing GC when scanning stock data takes too long #2470
  - Fix the issue that the changefeed update command does not recognize global command line parameters #2803