TiDB 5.0.6 Release Notes

Release date: December 31, 2021

TiDB version: 5.0.6

Compatibility changes

  • Tools

    • TiCDC

      • Change the output of the cdc server command error from stdout to stderr #3133
      • Set the default value of Kafka sink max-message-bytes to 10M #3081
      • Change the default value of Kafka Sink partition-num to 3 so that TiCDC distributes messages across Kafka partitions more evenly #3337

Improvements

  • TiDB

    • Show the affected SQL statements in the debug log when the coprocessor encounters a lock, which is helpful in diagnosing problems #27718

  • TiKV

    • Increase the speed of inserting SST files by moving the verification process to the Import thread pool from the Apply thread pool #11239
    • Add more metrics for the garbage collection module of Raft logs to locate performance problems in the module #11374
    • Collapse some uncommon storage-related metrics in Grafana dashboard #11681

  • PD

    • Speed up the exit process of schedulers #4146
    • Make the scheduling results of the scatter-range-scheduler scheduler more even by allowing the scheduler to schedule empty Regions and fix the configurations of the scheduler #4497
    • Support that the evict leader scheduler can schedule Regions with unhealthy peers #4093

  • Tools

    • TiCDC

      • Optimize rate limiting control on TiKV reloads to reduce gPRC congestion during changefeed initialization #3110
      • Add a tick frequency limit to EtcdWorker to prevent frequent etcd writes from affecting PD services #3112
      • Add the default configuration for config.Metadata.Timeout in Kafka sink #3352
      • Set the default value of max-message-bytes to 10M, to reduce the probability that Kafka messages cannot be sent #3081
      • Add more Prometheus and Grafana monitoring metrics and alerts, including no owner alert, mounter row, table sink total row, and buffer sink total row #4054 #1606

    • Backup & Restore (BR)

      • Retry BR tasks when encountering the PD request error or the TiKV I/O timeout error #27787
      • Improve the robustness of restoring #27421

    • TiDB Lightning

      • Support importing data into tables that have expression index or the index that depends on virtual generated columns #1404

Bug fixes

  • TiDB

    • Fix the issue that optimistic transaction conflicts might cause transactions to block each other #11148
    • Fix the issue of false positive error log invalid cop task execution summaries length for MPP queries #1791
    • Fix the panic that might occur when DML and DDL statements are executed concurrently #30940
    • Fix the privilege check fail error when performing the grant and revoke operations to grant and revoke global level privileges #29675
    • Fix the TiDB panic when executing the ALTER TABLE.. ADD INDEX statement in some cases #27687
    • Fix the issue that the enforce-mpp configuration does not take effect in v5.0.4 #29252
    • Fix the panic when using the CASE WHEN function on the ENUM data type #29357
    • Fix wrong results of the microsecond function in vectorized expressions #29244
    • Fix the issue of incomplete log information from the auto analyze result #29188
    • Fix wrong results of the hour function in vectorized expression #28643
    • Fix the unexpected error like tidb_cast to Int32 is not supported when the unsupported cast is pushed down to TiFlash #23907
    • Fix a bug that the availability detection of MPP node does not work in some corner cases #3118
    • Fix the DATA RACE issue when assigning MPP task ID #27952
    • Fix the INDEX OUT OF RANGE error for a MPP query after deleting an empty dual table #28250
    • Fix the TiDB panic when inserting invalid date values concurrently #25393
    • Fix the unexpected can not found column in Schema column error for queries in the MPP mode #30980
    • Fix the issue that TiDB might panic when TiFlash is shutting down #28096
    • Fix the unexpected index out of range error when the planner is doing join reorder #24095
    • Fix wrong results of the control functions (such as IF and CASE WHEN) when using the ENUM type data as parameters of such functions #23114
    • Fix the wrong result of CONCAT(IFNULL(TIME(3)) #29498
    • Fix wrong results of GREATEST and LEAST when passing in unsigned BIGINT arguments #30101
    • Fix the issue that a SQL operation is canceled when its JSON type column joins its CHAR type column #29401
    • Fix the data inconsistency issue caused by incorrect usage of lazy existence check and untouched key optimization #30410
    • Fix the issue that window functions might return different results when using a transaction or not #29947
    • Fix the issue that the SQL statements that contain cast(integer as char) union string return wrong results #29513
    • Fix the issue that the length information is wrong when casting Decimal to String #29417
    • Fix the issue that the Column 'col_name' in field list is ambiguous error is reported unexpectedly when a SQL statement contains natural join #25041
    • Fix the issue that the GREATEST function returns inconsistent results due to different values of tidb_enable_vectorized_expression (set to on or off) #29434
    • Fix the issue that the planner might cache invalid plans for join in some cases #28087
    • Fix the issue that the index out of range [1] with length 1 error is reported when a SQL statement evaluates an aggregation result on the result of join in some cases #1978

  • TiKV

    • Fix the issue that a down TiKV node causes the resolved timestamp to lag #11351
    • Fix the issue that batch messages are too large in Raft client implementation #9714
    • Fix a panic issue that occurs when Region merge, ConfChange, and Snapshot happen at the same time in extreme conditions #11475
    • Fix the issue that TiKV cannot detect the memory lock when TiKV perform a reverse table scan #11440
    • Fix the issue of negative sign when the decimal divide result is zero #29586
    • Fix the issue that the accumulation of GC tasks might cause TiKV to be OOM (out of memory) #11410
    • Fix the issue that the average latency of the by-instance gRPC requests is inaccurate in TiKV metrics #11299
    • Fix a memory leak caused by monitoring data of statistics threads #11195
    • Fix the issue of TiCDC panic that occurs when the downstream database is missing #11123
    • Fix the issue that TiCDC adds scan retries frequently due to the Congest error #11082
    • Fix the issue that the Raft connection is broken when the channel is full #11047
    • Fix the issue of TiKV panic that occurs when the files do not exist when TiDB Lightning imports data #10438
    • Fix the issue that TiDB cannot correctly identify whether the Int64 types in Max/Min functions are a signed integer or not, which causes the wrong calculation result of Max/Min #10158
    • Fix the issue that the node of a TiKV replica is down after the node gets snapshots because TiKV cannot modify the metadata accurately #10225
    • Fix the leak issue of the backup thread pool #10287
    • Fix the issue of casting illegal strings into floating-point numbers #23322

  • PD

    • Fix a panic issue that occurs after the TiKV node is removed #4344
    • Fix the issue that operator can get blocked due to down store #3353
    • Fix slow leader election caused by stuck Region syncer #3936
    • Fix the issue that the speed of removing peers is limited when repairing the down nodes #4090
    • Fix the issue that the hotspot cache cannot be cleared when the Region heartbeat is less than 60 seconds #4390

  • TiFlash

    • Fix potential data inconsistency after altering a primary key column to a larger int data type
    • Fix the issue that TiFlash fails to start up on some platforms, such as ARM, due to the absence of the libnsl.so library
    • Fix the issue that the Store size metric does not match the actual data size on a disk
    • Fix the issue that TiFlash crashes due to a Cannot open file error
    • Fix occasional crashes of TiFlash when an MPP query is killed
    • Fix the unexpected error 3rd arguments of function substringUTF8 must be constants
    • Fix query failures caused by excessive OR conditions
    • Fix the bug that results of where <string> are wrong
    • Fix inconsistent behaviors of CastStringAsDecimal between TiFlash and TiDB/TiKV
    • Fix query failures caused by the error different types: expected Nullable(Int64), got Int64
    • Fix query failures caused by the error Unexpected type of column: Nullable(Nothing)
    • Fix query failures caused by overflow when comparing data in the DECIMAL data type

  • Tools

    • TiCDC

      • Fix the issue that some partitioned tables without valid indexes might be ignored when force-replicate is enabled #2834
      • Fix the issue that cdc cli silently truncates user parameters when receiving unexpected parameters, causing the user input parameters to be lost #2303
      • Fix the issue that TiCDC sync task might pause when an error occurs during writing a Kafka message #2978
      • Fix a possible panic issue when encoding some types of columns into Open Protocol format #2758
      • Fix the issue that Kafka may send excessively large messages by setting the default value of max-message-bytes to 10M #3081
      • Fix the issue that TiCDC replication task might terminate when the upstream TiDB instance unexpectedly exits #3061
      • Fix the issue that TiCDC process might panic when TiKV sends duplicate requests to the same Region #2386
      • Fix the TiCDC replication interruption issue when multiple TiKVs crash or during a forced restart #3288
      • Fix the negative value error in the changefeed checkpoint lag #3010
      • Fix the issue of overly frequent warnings caused by MySQL sink deadlock #2706
      • Fix the issue that Avro sink does not support parsing JSON type columns #3624
      • Fix the bug that TiCDC reads the incorrect schema snapshot from TiKV when the TiKV owner restarts #2603
      • Fix the memory leak issue after processing DDLs #3174
      • Fix the bug that the enable-old-value configuration item is not automatically set to true on Canal and Maxwell protocols #3676
      • Fix the timezone error that occurs when the cdc server command runs on some Red Hat Enterprise Linux releases (such as 6.8 and 6.9) #3584
      • Fix the issue of the inaccurate txn_batch_size monitoring metric for Kafka sink #3431
      • Fix the issue that tikv_cdc_min_resolved_ts_no_change_for_1m keeps alerting when there is no changefeed #11017
      • Fix the TiCDC panic issue that occurs when manually cleaning the task status in etcd #2980
      • Fix the issue that changefeed does not fail fast enough when the ErrGCTTLExceeded error occurs #3111
      • Fix the issue that scanning stock data might fail due to TiKV performing GC when scanning stock data takes too long #2470
      • Fix OOM in container environments #1798

    • Backup & Restore (BR)

      • Fix a bug that the average speed is inaccurately calculated for backup and restore #1405

    • Dumpling

      • Fix the bug that Dumpling becomes very slow when dumping tables with the composite primary key or unique key #29386