Elasticsearch version 7.10.0
- Elasticsearch version 7.10.0

Elasticsearch version 7.10.0

Also see Breaking changes in 7.10.

Security updates

Elasticsearch versions before 7.10.0 and 6.8.14 have an information disclosure issue when audit logging and the emit_request_body option are enabled. The Elasticsearch audit log could contain sensitive information, such as password hashes or authentication tokens. This could allow an Elasticsearch administrator to view these details. You must upgrade to Elasticsearch version 7.10.0 to obtain the fix. CVE-2020-7021

Known issues

SQL: If a WHERE clause contains at least two relational operators joined by AND, of which one is a comparison (<=, <, >=, >) and another one is an inequality (!=, <>), both against literals or foldable expressions, the inequality will be ignored. The workaround is to substitute the inequality with a NOT IN operator.

We have fixed this issue in Elasticsearch 7.10.1 and later versions. For more details, see #65488.
There were several regressions around loading nested documents. These have been addressed in Elasticsearch 7.10.2.
- With multiple layers of nested inner_hits, we can fail to load the _source. (#66577)
- With nested inner_hits, the fast vector highlighter may load snippets from the wrong document. (#65533)
- When _source is disabled, we can fail load nested inner_hits and top_hits. (#66572)
Snapshot and restore: If an index is deleted while the cluster is concurrently taking more than one snapshot then there is a risk that one of the snapshots may never complete and also that some shard data may be lost from the repository, causing future restore operations to fail. To mitigate this problem, set snapshot.max_concurrent_operations: 1 to prevent concurrent snapshot operations:
```
PUT _cluster/settings
{
  "persistent" : {
    "snapshot.max_concurrent_operations" : 1
  }
}
```
This issue is fixed in Elasticsearch versions 7.13.1 and later. It is not possible to repair a repository once it is affected by this issue, so you must restore the repository from a backup, or clear the repository by executing DELETE _snapshot/<repository name>/*, or move to a fresh repository. For more details, see #73456.
Parsing a request when the last element in an array is filtered out (for instance using _source_includes) fails. This is due to a bug in Jackson parser. Fixed in Elasticsearch 8.6.1 (#91456)
The deprecated index.mapper.dynamic setting can break your cluster. It can only be set using the Update index settings API. Symptoms include nodes failing to start or shards failing to allocate. Do not use this setting in versions prior to 7.17.22. The bug is fixed in 7.17.22. (issue: #109160)

Breaking changes

Authentication

API key name should always be required for creation #59836 (issue: #59484)

Network

Set specific keepalive options by default on supported platforms #59278

Breaking Java changes

Machine Learning

Deprecate allow_no_jobs and allow_no_datafeeds in favor of allow_no_match #60601 (issue: #60642)

Mapping

Pass SearchLookup supplier through to fielddataBuilder #61430 (issue: #59332)

Deprecations

Cluster Coordination

Deprecate and ignore join timeout #60872 (issue: #60873)

Machine learning

Renames */inference* APIs to */trained_models* #63097

New features

Aggregations

Add rate aggregation #61369 (issue: #60674)

Features/Features

Add data tiers (hot, warm, cold, frozen) as custom node roles #60994 (issue: #60848)
Allocate newly created indices on data_hot tier nodes #61342 (issue: #60848)

Features/ILM+SLM

ILM migrate data between tiers #61377 (issue: #60848)
ILM: add force-merge step to searchable snapshots action #60819 (issues: #53488, #56215)

Machine Learning

Implement AucRoc metric for classification #60502 (issue: #62160)

Mapping

Introduce 64-bit unsigned long field type #60050 (issue: #32434)

Add search fields option to support high-level field retrieval #60100 (issues: #49028, #55363)

Enhancements

Aggregations

Adds hard_bounds to histogram aggregations #59175 (issue: #50109)
Allocate slightly less per bucket #59740
Improve reduction of terms aggregations #61779 (issue: #51857)
Speed up date_histogram by precomputing ranges #61467

Analysis

Support ignore_keywords flag for word delimiter graph token filter #59563 (issue: #59491)

Authentication

Cache API key doc to reduce traffic to the security index #59376 (issues: #53940, #55836)
Include authentication type for the authenticate response #61247 (issue: #61130)
Oidc additional client auth types #58708
Warn about unlicensed realms if no auth token can be extracted #61402 (issue: #61090)

Authorization

Add DEBUG logging for undefined role mapping field #61246 (issue: #48562)
Add more context to index access denied errors #60357 (issue: #42166)

CRUD

Add configured indexing memory limit to node stats #60342
Dedicated threadpool for system index writes #61655

Cluster Coordination

Add more useful toString on cluster state observers #60277
Fail invalid incremental cluster state writes #61030
Provide option to allow writes when master is down #60605

Distributed

Detect noop of update index settings #61348
Thread safe clean up of LocalNodeModeListeners #59932 (issue: #59801)

Features/CAT APIs

Adding Hit counts and Miss counts for QueryCache exposed through REST API #60114 (issue: #48645)

Features/Features

Add aggregation list to node info #60074 (issue: #52057)
Adding new require_alias option to indexing requests #58917 (issue: #55267)

Features/ILM+SLM

Move internal index templates to composable templates #61457

Features/Ingest

Add network from MaxMind Geo ASN database #61676
Allow_duplicates option for append processor #61916 (issue: #57543)
Configurable output format for date processor #61324 (issue: #42523)
Enhance the ingest node simulate verbose output #60433 (issue: #56004)
Per processor description for verbose simulate #58207 (issue: #57906)
Preserve grok pattern ordering and add sort option #61671 (issue: #40819)

Features/Java High Level REST Client

HLRC: UpdateByQuery API with wait_for_completion being false #58552 (issues: #35202, #46350)

Infra/Core

Add logstash system index APIs #53350
Deprecate REST access to System Indices #60945
Speed up Compression Logic by Pooling Resources #61358
System index reads in separate threadpool #57936 (issues: #37867, #50251)

Infra/Logging

Do not create two loggers for DeprecationLogger #58435
Header warning logging refactoring #55941 (issues: #52369, #55699)
Write deprecation logs to a data stream #61484 (issues: #46106, #61474)

Infra/Packaging

Add UBI docker builds #60742
Upgrade Centos version in Dockerfile to 8 #59019

Infra/Resiliency

Remove node from cluster when node locks broken #61400 (issues: #52680, #58373)

Infra/Scripting

Augment String with sha1 and sha256 #59671 (issue: #59633)
Converts casting and def support #61350 (issue: #59647)

Machine Learning

Add a “verbose” option to the data frame analytics stats endpoint #59589 (issue: #59125)
Add new include flag to get trained models API for model training metadata #61922
Add new feature_processors field for data frame analytics #60528 (issue: #59327)
Add new n_gram_encoding custom processor #61578
During nightly maintenance delete jobs whose original deletion tasks were lost #60121 (issue: #42840)
Suspend persistence of trained model stats when ML upgrade mode is enabled #61143
Calculate total feature importance to store with model metadata #1387
Change outlier detection feature_influence format to array with nested objects #1475, #62068
Add timeouts to named pipe connections #1514, #62993 (issue: #1504)

Mapping

Add field type for version strings #59773 (issue: #48878)
Allow [null] values in [null_value] #61798 (issues: #7978, #58823)
Allow metadata fields in the _source #61590 (issue: #58339)

Network

Improve deserialization failure logging #60577 (issue: #38939)
Log and track open/close of transport connections #60297

Performance

Speed up empty highlighting many fields #61860

SQL

Add option to provide the delimiter for the CSV format #59907 (issue: #41634)
Implement DATE_PARSE function for parsing strings into DATE values #57391 (issue: #54962)
Implement FORMAT function #55454 (issue: #54965)

Avoid reloading _source for every inner hit #60494 (issue: #32818)
Cancel multisearch when http connection closed #61399
Enable cancellation for msearch requests #61337
Executes incremental reduce in the search thread pool #58461 (issues: #51857, #53411)
Introduce point in time APIs in x-pack basic #61062 (issues: #26472, #46523)
ParametrizedFieldMapper to run validators against default value #60042 (issue: #59332)
Add case insensitive flag for “term” family of queries #61596 (issue: #61546)
Add case insensitive support for regex queries #59441
Tweak toXContent implementation of ParametrizedFieldMapper #59968
Implement fields value fetching for the text, search_as_you_type and token_count field types #63515
Make term/prefix/wildcard/regex query parsing more lenient, with respect to the case_insensitive flag #63926 (issue: #63893)

Snapshot/Restore

Add repositories metering API #60371
Clone Snapshot API #61839
Determine shard size before allocating shards recovering from snapshots #61906
Introduce index based snapshot blob cache for Searchable Snapshots #60522
Validate snapshot UUID during restore #59601 (issue: #50999)

Store

Report more details of unobtainable ShardLock #61255 (issue: #38807)

Transform

Add support for missing bucket #59591 (issues: #42941, #55102)

Bug fixes

Aggregations

Fix AOOBE when setting min_doc_count to 0 in significant_terms #60823 (issues: #60683, #60824)
Make sure non-collecting aggs include sub-aggs #64214 (issue: #64142)
Composite aggregation must check live docs when the index is sorted #63864
Fix broken parent and child aggregator #63811

Allocation

Fix scheduling of ClusterInfoService#refresh #59880

Authorization

Fix doc-update interceptor for indices with DLS and FLS #61516
Report anonymous roles in authenticate response #61355 (issues: #47195, #53453, #57711, #57853)
Add view_index_metadata privilege over metricbeat-* for monitoring agent #63750 (issue: #63750)

CRUD

Propagate forceExecution when acquiring permit #60634 (issue: #60359)

Cluster Coordination

Reduce allocations when persisting cluster state #61159

Distributed

Fix cluster health rest api wait_for_no_initializing_shards bug #58379
Fix cluster health when closing #61709

Engine

Fix estimate size of translog operations #59206

Features/ILM+SLM

Fix ILM history index settings #61880 (issues: #61457, #61863)
Ensure cancelled SLM jobs do not continue to run #63762 (issue: #63754)

Features/Java Low Level REST Client

Handle non-default port in Cloud-Id #61581

Features/Stats

Remove sporadic min/max usage estimates from stats #59755

Features/Watcher

Correct the query dsl for watching elasticsearch version #58321 (issue: #58261)
Fix passing params to template or script failed in watcher #58559 (issue: #57625)

Geo

Fix wrong NaN comparison #61795 (issue: #48207)

Infra/Core

Throws IndexNotFoundException in TransportGetAction for unknown System indices #61785 (issue: #57936)
Handle missing logstash index exceptions #63698
XPack Usage API should run on MANAGEMENT threads #64160

Infra/Packaging

Allow running the Docker image with a non-default group #61194 (issue: #60864)
Set the systemd initial timeout to 75 seconds #60345 (issue: #60140)

Machine Learning

Adjusting inference processor to support foreach usage #60915 (issue: #60867)
Get data frame analytics jobs stats API can return multiple responses if more than one error #60900 (issue: #60876)
Do not mark the data frame analytics job as FAILED when a failure occurs after the node is shutdown #61331 (issue: #60596)
Improve handling of exception while starting data frame analytics process #61838 (issue: #61704)
Fix progress on resume after final training has completed for classification and regression. Previously, progress was shown stuck at zero for final training. #1443
Avoid potential “Failed to compute quantile” and “No values added to quantile sketch” log errors training regression and classification models if there are features with mostly missing values #1500
Correct the anomaly detection job model state min_version #1546

Mapping

Improve ignore_malformed handling for dates #60211 (issue: #52634)

Network

Let isInetAddress utility understand the scope ID on ipv6 #60172 (issue: #60115)
Suppress noisy SSL exceptions #61359

Allows nanosecond resolution in search_after #60328 (issue: #52424)
Consolidate validation for docvalue_fields #59473
Correct how field retrieval handles multifields and copy_to #61309 (issue: #61033)
Apply boost only once for distance_feature query #63767
Fixed NullPointerException in significant_text aggregation when field does not exist #64144 (issue: #64045)
Fix async search to retry updates on version conflict #63652 (issue: #63213)
Fix sorted query when date_nanos is used as the numeric_type #64183 (issue: #63719)

Snapshot/Restore

Avoid listener call under SparseFileTracker#mutex #61626 (issue: #61520)
Ensure repo not in use for wildcard repo deletes #60947
Fix Test Failure in testCorrectCountsForDoneShards #60254 (issue: #60247)
Minimize cache file locking during prewarming #61837 (issue: #58658)
Prevent snapshots to be mounted as system indices #61517 (issue: #60522)
Make Searchable Snapshot’s CacheFile Lock less #63911 (issue: #63586)
Don’t Generate an Index Setting History UUID unless it’s supported #64213 (issue: #64152)

SQL

Allow unescaped wildcard (*) in LIKE pattern #63428 (issue: #55108)
Validate integer paramete in string functions #63728 (issue: #58923)
Remove filter from field_caps requests #63840 (issue: #63832)

Upgrades

Infra/Packaging

Upgrade bundled JDK to 15.0.1 and switch to AdoptOpenJDK #64253

Store