TiDB 4.0.5 Release Notes

Release date: August 31, 2020

TiDB version: 4.0.5

Compatibility Changes

  • TiDB

    • Change drop partition and truncate partition‘s job arguments to support the ID array of multiple partitions #18930
    • Add the delete-only state for checking add partition replicas #18865

New Features

  • TiKV

    • Define error code for errors #8387

  • TiFlash

    • Support the unified log format with TiDB

  • Tools

    • TiCDC

      • Support Kafka SSL connection #764
      • Support outputting the old value #708
      • Add the column flags #796
      • Support outputting the DDL statements and table schema of the previous version #799

Improvements

  • TiDB

    • Optimize the performance of DecodePlan for big union queries #18941
    • Reduce the number of GC lock scans when the Region cache miss error occurs #18876
    • Ease the impact of statistical feedback on cluster performance #18772
    • Support canceling operations before the RPC response is returned #18580
    • Add the HTTP API to generate the TiDB metric profile #18531
    • Support scattering partitioned tables #17863
    • Add detailed memory usage of each instance in Grafana #18679
    • Show the detailed runtime information of the BatchPointGet operator in the result of EXPLAIN #18892
    • Show the detailed runtime information of the PointGet operator in the result of EXPLAIN #18817
    • Warn the potential deadlock for Consume in remove() #18395
    • Refine the behaviors of StrToInt and StrToFloat and support converting JSON to the date, time, and timestamp types #18159
    • Support limiting the memory usage of the TableReader operator #18392
    • Avoid too many times of backoff when retrying the batch cop request #18999
    • Improve compatibility for ALTER TABLE algorithms #19270
    • Make the single partitioned table support IndexJoin on the inner side #19151
    • Support searching the log file even when the log includes invalid lines #18579

  • PD

    • Support scattering Regions in stores with special engines (such as TiFlash) #2706
    • Support the Region HTTP API to prioritize Region scheduling of a given key range #2687
    • Improve the leader distribution after Region scattering #2684
    • Add more tests and logs for the TSO request #2678
    • Avoid invalid cache updates after the leader of a Region has changed #2672
    • Add an option to allow store.GetLimit to return the tombstone stores #2743
    • Support synchronizing the Region leader change between the PD leader and followers #2795
    • Add commands for querying the GC safepoint service #2797
    • Replace the region.Clone call in filters to improve performance #2801
    • Add an option to disable updating Region flow cache to improve the performance of the large cluster #2848

  • TiFlash

    • Add more Grafana panels to display metrics of CPU, I/O, RAM usages and metrics of the storage engine
    • Reduce I/O operations by optimizing the processing logic of Raft logs
    • Accelerate Region scheduling for the blocked add partition DDL statement
    • Optimize compactions of delta data in DeltaTree to reduce read and write amplification
    • Optimize the performance of applying Region snapshots by preprocessing the snapshots using multiple threads
    • Optimize the number of opening file descriptors when the read load of TiFlash is low to reduce system resource consumption
    • Optimize the number of unnecessary small files created when TiFlash restarts
    • Support encryption at rest for data storage
    • Support TLS for data transfer

  • Tools

    • TiCDC

      • Lower the frequency of getting TSO #801

    • Backup & Restore (BR)

      • Optimize some logs #428

    • Dumpling

      • Release FTWRL after connections are created to reduce the lock time for MySQL #121

    • TiDB Lightning

      • Optimize some logs #352

Bug Fixes

  • TiDB

    • Fix the should ensure all columns have the same length error that occurs because the ErrTruncate/Overflow error is incorrectly handled in the builtinCastRealAsDecimalSig function #18967
    • Fix the issue that the pre_split_regions table option does not work in the partitioned table #18837
    • Fix the issue that might cause a large transaction to be terminated prematurely #18813
    • Fix the issue that using the collation functions get wrong query results #18735
    • Fix the bug that the getAutoIncrementID() function does not consider the tidb_snapshot session variable, which might cause the dumper tool to fail with the table not exist error #18692
    • Fix the unknown column error for SQL statement like select a from t having t.a #18434
    • Fix the panic issue that writing the 64-bit unsigned type into the hash partitioned table causes overflow and gets an unexpected negative number when the partition key is the integer type #18186
    • Fix the wrong behavior of the char function #18122
    • Fix the issue that the ADMIN REPAIR TABLE statement cannot parse integer in the expressions on the range partition #17988
    • Fix the wrong behavior of the SET CHARSET statement #17289
    • Fix the bug caused by the wrong collation setting which leads to the wrong result of the collation function #17231
    • Fix the issue that STR_TO_DATE‘s handling of the format tokens ‘%r’, ‘%h’ is inconsistent with that of MySQL #18727
    • Fix issues that the TiDB version information is inconsistent with that of PD/TiKV in the cluster_info table #18413
    • Fix the existent checks for pessimistic transactions #19004
    • Fix the issue that executing union select for update might cause concurrent race #19006
    • Fix the wrong query result when apply has a child of the PointGet operator #19046
    • Fix the incorrect result that occurs when IndexLookUp is in the inner side of the Apply operator #19496
    • Fix the incorrect result of anti-semi-join queries #19472
    • Fix the incorrect result caused by the mistaken usage of BatchPointGet #19456
    • Fix the incorrect result that occurs when UnionScan is in the inner side of the Apply operator #19496
    • Fix the panic caused by using the EXECUTE statement to print an expensive query log #17419
    • Fix the index join error when the join key is ENUM or SET #19235
    • Fix the issue that the query range cannot be built when the NULL value exists on the index column #19358
    • Fix the data race issue caused by updating the global configuration #17964
    • Fix the panic issue occurs when modifying the character set in an uppercase schema #19286
    • Fix an unexpected error caused by changing the temporary directory during the disk spill action #18970
    • Fix the wrong hash key for the decimal type #19131
    • Fix the issue that the PointGet and BatchPointGet operators do not consider the partition selection syntax and get incorrect results #19141
    • Fix the incorrect results when using the Apply operator together with the UnionScan operator #19104
    • Fix the bug that causes the indexed virtual generated column to return wrong value #17989
    • Add the lock for runtime statistics to fix a panic caused by concurrent execution #18983

  • TiKV

    • Speed up leader election when Hibernate Region is enabled #8292
    • Fix the memory leak issue during scheduling #8357
    • Add the hibernate-timeout configuration item to prevent the leader from becoming hibernate too fast #8208

  • PD

    • Fix the bug that the TSO request might fail at the time of leader change #2666
    • Fix the issue that sometimes Region replicas cannot be scheduled to the optimal state when placement rules are enabled #2720
    • Fix the issue that Balance Leader does not work when placement rules are enabled #2726
    • Fix the issue that unhealthy stores are not filtered from store load statistics #2805

  • TiFlash

    • Fix the issue that TiFlash cannot start normally after upgrading from an earlier version if the name of the database or table contains special characters
    • Fix the issue that the TiFlash process cannot exit if any exceptions are thrown during initialization

  • Tools

    • Backup & Restore (BR)

      • Fix the issue of duplicated calculation of total KV and total bytes in the backup summary log #472
      • Fix the issue that the import mode does not work in the first 5 minutes after switching to this mode #473

    • Dumpling

      • Fix the issue that FTWRL lock is not released in time #128

    • TiCDC

      • Fix the issue that the failed changefeed cannot be removed #782
      • Fix invalid delete events by selecting one unique index as the handle index #787
      • Fix the bug that GC safepoint is forwarded beyond the checkpoint of stopped changefeed #797
      • Fix the bug that the network I/O waiting blocks tasks to exit #825
      • Fix the bug that some unnecessary data might be mistakenly replicated to the downstream #743

    • TiDB Lightning

      • Fix the syntax error on empty binary/hex literals when using TiDB backend #357