Elasticsearch version 7.0.0

These release notes include all changes made in the alpha, beta, and RC releases of 7.0.0.

Also see Breaking changes in 7.0.

Known issues

• Applying deletes or updates on an index after it has been shrunk may corrupt the index. In order to prevent this issue, it is recommended to stop shrinking read-write indices. For read-only indices, it is recommended to force-merge indices after shrinking, which significantly reduces the likeliness of this corruption in the case that deletes/updates would be applied by mistake. This bug is fixed in Elasticsearch 7.7 and later versions. More details can be found on the corresponding issue.
• Indices created in 6.x with date and date_nanos fields using formats that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results. https://github.com/elastic/elasticsearch/pull/52555 This is fixed in Elasticsearch 7.7 and later versions.

Breaking changes

Aggregations

• Remove support for deprecated params._agg/_aggs for scripted metric aggregations #32979 (issues: #29328, #31597)
• Percentile/Ranks should return null instead of NaN when empty #30460 (issue: #29066)
• Render sum as zero if count is zero for stats aggregation #27193 (issue: #26893)

Analysis

• Remove delimited_payload_filter #27705 (issues: #26625, #27704)
• Limit the number of tokens produced by _analyze #27529 (issue: #27038)
• Add limits for ngram and shingle settings #27211 (issue: #25887)

Audit

• Logfile auditing settings remove after deprecation #35205
• Remove index audit output type #37707 (issues: #29881, #37301)

Authentication

• Security: remove wrapping in put user response #33512 (issue: #32332)
• Remove bwc logic for token invalidation #36893 (issue: #36727)

Authorization

• Remove aliases resolution limitations when security is enabled #31952 (issue: #31516)
• Remove implicit index monitor privilege #37774

Circuit Breakers

• Lower fielddata circuit breaker’s default limit #27162 (issue: #27130)

CRUD

• Version conflict exception message enhancement #29432 (issue: #21278)
• Using ObjectParser in UpdateRequest #29293 (issue: #28740)
• Remove support for internal versioning for concurrency control #38254 (issue: #1078)

Distributed

• Remove undocumented action.master.force_local setting #29351
• Remove tribe node support #28443
• Forbid negative values for index.unassigned.node_left.delayed_timeout #26828
• Remove cluster state size #40061 (issues: #39806, #39827, #39951, #40016)

Features/Features

• Remove Migration Upgrade and Assistance APIs #40075 (issue: #40014)

Features/Indices APIs

• Indices Exists API should return 404 for empty wildcards #34499
• Default to one shard #30539
• Limit the number of nested documents #27405 (issue: #26962)

Features/Ingest

• Add Configuration Except. Data to Metdata #32322 (issue: #27728)
• Add ECS schema for user-agent ingest processor (#37727) #37984 (issues: #37329, #37727)
• Remove special handling for ingest plugins #36967 (issues: #36898, #36956)

Features/Java Low Level REST Client

• Drop support for the low-level REST client on JDK 7 #38540 (issue: #29607)

Features/Watcher

• Remove Watcher Account “unsecure” settings #36736 (issue: #36403)

Features/Stats

• Remove the suggest metric from stats APIs #29635 (issue: #29589)
• Align cat thread pool info to thread pool config #29195 (issue: #29123)
• Align thread pool info to thread pool configuration #29123 (issue: #29113)

Geo

• Use geohash cell instead of just a corner in geo_bounding_box #30698 (issue: #25154)

Index APIs

• Always enforce cluster-wide shard limit #34892 (issues: #20705, #34021)

Infra/Circuit Breakers

• Introduce durability of circuit breaking exception #34460 (issue: #31986)
• Circuit-break based on real memory usage #31767

Infra/Core

• Default node.name to the hostname #33677
• Remove bulk fallback for write thread pool #29609
• CCS: Drop http address from remote cluster info #29568 (issue: #29207)
• Remove the index thread pool #29556
• Main response should not have status 503 when okay #29045 (issue: #8902)
• Automatically prepare indices for splitting #27451
• Don’t refresh on _flush _force_merge and _upgrade #27000 (issue: #26972)

Infra/Logging

• Elasticsearch json logging #36833 (issue: #32850)

Infra/Packaging

• Packaging: Remove windows bin files from the tar distribution #30596
• Package ingest-user-agent as a module #36956
• Package ingest-geoip as a module #36898

Infra/REST API

• Remove GET support for clear cache indices #29525
• Clear Indices Cache API remove deprecated url params #29068

Infra/Scripting

• Remove support for deprecated StoredScript contexts #31394 (issues: #27612, #28939)
• Remove getDate methods from ScriptDocValues #30690
• Drop ScriptDocValues#date and ScriptDocValues#dates in 7.0.0 #30690 (issue: #23008)

Infra/Settings

• Remove config prompting for secrets and text #27216

Machine Learning

• Remove types from datafeed #36538 (issue: #34265)

Mapping

• Match phrase queries against non-indexed fields should throw an exception #31060
• Remove legacy mapping code. #29224
• Reject updates to the _default_ mapping. #29165 (issues: #15613, #28248)
• Remove the update_all_types option. #28288
• Remove the _default_ mapping. #28248
• Reject the index_options parameter for numeric fields #26668 (issue: #21475)
• Update the default for include_type_name to false. #37285
• Support include_type_name in RestGetIndicesAction #37149

Network

• Remove http.enabled setting #29601 (issue: #12792)
• Remove HTTP max content length leniency #29337
• Remove TLS 1.0 as a default SSL protocol #37512 (issue: #36021)
• Security: remove SSL settings fallback #36846 (issue: #29797)

Percolator

• Remove deprecated percolator map_unmapped_fields_as_string setting #28060

Ranking

• Add minimal sanity checks to custom/scripted similarities. #33564 (issue: #33309)
• Scroll queries asking for rescore are considered invalid #32918 (issue: #31775)
• Forbid negative scores in function_score query #35709 (issue: #33309)
• Forbid negative field boosts in analyzed queries #37930 (issue: #33309)

Scripting

• Delete deprecated getValues from ScriptDocValues #36183 (issue: #22919)

Search

• Remove deprecated url parameters _source_include and _source_exclude #35097 (issues: #22792, #33475)
• Disallow negative query boost #34486 (issue: #33309)
• Forbid negative weight in Function Score Query #33390 (issue: #31927)
• In the field capabilities API, remove support for providing fields in the request body. #30185
• Remove deprecated options for query_string #29203 (issue: #25551)
• Fix Laplace scorer to multiply by alpha (and not add) #27125
• Remove _primary and _replica shard preferences #26791 (issue: #26335)
• Limit the number of expanded fields it query_string and simple_query_string #26541 (issue: #25105)
• Make purely negative queries return scores of 0. #26015 (issue: #23449)
• Remove the deprecated _termvector endpoint. #36131 (issues: #36098, #8484)
• Remove deprecated Graph endpoints #35956
• Validate metadata on _msearch #35938 (issue: #35869)
• Make hits.total an object in the search response #35849 (issue: #33028)
• Remove the distinction between query and filter context in QueryBuilders #35354 (issue: #35293)
• Throw a parsing exception when boost is set in span_or query (#28390) #34112 (issue: #28390)
• Track total hits up to 10,000 by default #37466 (issue: #33028)
• Use mappings to format doc-value fields by default. #30831 (issues: #26948, #29639)

Security

• Remove heuristics that enable security on trial licenses #38075 (issue: #38009)

Snapshot/Restore

• Include size of snapshot in snapshot metadata #30890 (issue: #18543)
• Remove azure deprecated settings #26099 (issue: #23405)

Store

• Drop elasticsearch-translog for 7.0 #33373 (issues: #31389, #32281)
• completely drop index.shard.check_on_startup: fix for 7.0 #33194

Suggesters

• Fix threshold frequency computation in Suggesters #34312 (issue: #34282)
• Make Geo Context Mapping Parsing More Strict #32821 (issues: #32202, #32412)
• Remove the ability to index or query context suggestions without context #31007 (issue: #30712)

ZenDiscovery

• Best-effort cluster formation if unconfigured #36215
• Remove DiscoveryPlugin#getDiscoveryTypes #38414 (issue: #38410)

Breaking Java changes

Aggregations

• Change GeoHashGrid.Bucket#getKey() to return String #31748 (issue: #30320)

Analysis

• Remove deprecated AnalysisPlugin#requriesAnalysisSettings method #32037 (issue: #32025)

Features/Java High Level REST Client

• Drop deprecated methods from Retry #33925
• Cluster health to default to cluster level #31268 (issue: #29331)
• Remove deprecated API methods #31200 (issue: #31069)

Features/Java Low Level REST Client

• Drop deprecated methods #33223 (issues: #29623, #30315)
• Remove support for maxRetryTimeout from low-level REST client #38085 (issues: #25951, #31834, #33342)

Geo

• Decouple geojson parse logic from ShapeBuilders #27212

Infra/Core

• Remove RequestBuilder from Action #30966
• Handle scheduler exceptions #38014 (issues: #28667, #36137, #37708)

Infra/Transport API

• Java api clean up: remove deprecated isShardsAcked #28311 (issues: #27784, #27819)

ZenDiscovery

• Make node field in JoinRequest private #36405

Deprecations

Aggregations

• Deprecate dots in aggregation names #31468 (issues: #17600, #19040)

Analysis

• Replace parameter unicodeSetFilter with unicode_set_filter #29215 (issue: #22823)
• Replace delimited_payload_filter by delimited_payload #26625 (issue: #21978)
• Deprecate Standard Html Strip Analyzer in master #26719 (issue: #4704)
• Remove nGram and edgeNGram token filter names (#38911) #39070 (issues: #30209, #38911)

Audit

• Deprecate index audit output type #37301 (issue: #29881)

Core

• Deprecate use of scientific notation in epoch time parsing #36691
• Add backcompat for joda time formats #36531

Cluster Coordination

• Deprecate size in cluster state response #39951 (issue: #39806)

Features/Indices APIs

• Default copy settings to true and deprecate on the REST layer #30598
• Reject setting index.optimize_auto_generated_id after version 7.0.0 #28895 (issue: #27600)

Features/Ingest

• Deprecate _type in simulate pipeline requests #37949 (issue: #37731)

Features/Java High Level REST Client

• Deprecate HLRC security methods #37883 (issues: #36938, #37540)
• Deprecate HLRC EmptyResponse used by security #37540 (issue: #36938)

Features/Watcher

• Deprecate xpack.watcher.history.cleaner_service.enabled #37782 (issue: #32041)
• deprecate types for watcher #37594 (issue: #35190)

Graph

• Deprecate types in _graph/explore calls. #40466

Infra/Core

• Deprecate negative epoch timestamps #36793
• Deprecate use of scientific notation in epoch time parsing #36691

Infra/Packaging

• Deprecate fallback to java on PATH #37990

Infra/Scripting

• Add types deprecation to script contexts #37554
• Deprecate _type from LeafDocLookup #37491
• Remove deprecated params.ctx #36848 (issue: #34059)

Infra/Transport API

• Deprecate the transport client in favour of the high-level REST client #27085

Machine Learning

• Deprecate X-Pack centric ML endpoints #36315 (issue: #35958)
• Adding ml_settings entry to HLRC and Docs for deprecation_info #38118
• Datafeed deprecation checks #38026 (issue: #37932)
• Remove “8” prefixes from file structure finder timestamp formats #38016
• Adjust structure finder for Joda to Java time migration #37306
• Resolve 7.0.0 TODOs in ML code #36842 (issue: #29963)

Mapping

• Deprecate type exists requests. #34663
• Deprecate types in index API #36575 (issues: #35190, #35790)
• Deprecate uses of _type as a field name in queries #36503 (issue: #35190)
• Deprecate types in update_by_query and delete_by_query #36365 (issue: #35190)
• For msearch templates, make sure to use the right name for deprecation logging. #36344
• Deprecate types in termvector and mtermvector requests. #36182
• Deprecate types in update requests. #36181
• Deprecate types in document delete requests. #36087
• Deprecate types in get, exists, and multi get. #35930
• Deprecate types in search and multi search templates. #35669
• Deprecate types in explain requests. #35611
• Deprecate types in validate query requests. #35575
• Deprecate types in count and msearch. #35421 (issue: #34041)
• Deprecate types in rollover index API #38039 (issue: #35190)
• Deprecate types in get field mapping API #37667 (issue: #35190)
• Deprecate types in the put mapping API. #37280 (issues: #29453, #37285)
• Support include_type_name in the field mapping and index template APIs. #37210
• Deprecate types in create index requests. #37134 (issues: #29453, #37285)
• Deprecate use of the _type field in aggregations. #37131 (issue: #36802)
• Deprecate reference to _type in lookup queries #37016 (issue: #35190)
• Deprecate the document create endpoint. #36863
• Deprecate types in index API #36575 (issues: #35190, #35790)
• Deprecate types in update APIs #36225

Migration

• Deprecate X-Pack centric Migration endpoints #35976 (issue: #35958)

Monitoring

• Deprecate /_xpack/monitoring/* in favor of /_monitoring/* #36130 (issue: #35958)

Rollup

• Re-deprecate xpack rollup endpoints #36451 (issue: #36044)
• Deprecate X-Pack centric rollup endpoints #35962 (issue: #35958)

Scripting

• Adds deprecation logging to ScriptDocValues#getValues. #34279 (issue: #22919)
• Conditionally use java time api in scripting #31441

Search

• Deprecate filtering on _type. #29468 (issue: #15613)
• Remove X-Pack centric graph endpoints #36010 (issue: #35958)
• Deprecate use of type in reindex request body #36823
• Add typless endpoints for get_source and exist_source #36426

Security

• Deprecate X-Pack centric license endpoints #35959 (issue: #35958)
• Deprecate /_xpack/security/* in favor of /_security/* #36293 (issue: #35958)

SQL

• Deprecate X-Pack SQL translate endpoint #36030
• Deprecate X-Pack centric SQL endpoints #35964 (issue: #35958)

Watcher

• Deprecate X-Pack centric watcher endpoints #36218 (issue: #35958)

New features

Allocation

• Node repurpose tool #39403 (issues: #37347, #37748)

Analysis

• Relax TermVectors API to work with textual fields other than TextFieldType #31915 (issue: #31902)
• Add support for inlined user dictionary in Nori #36123 (issue: #35842)
• Add a prebuilt ICU Analyzer #34958 (issue: #34285)

Authentication

• Add support for API keys to access Elasticsearch #38291 (issue: #34383)
• OIDC realm authentication flows #37787
• OIDC Realm JWT+JWS related functionality #37272 (issues: #35339, #37009)
• OpenID Connect Realm base functionality #37009 (issue: #35339)

Authorization

• Allow custom authorization with an authorization engine #38358 (issues: #32435, #36245, #37328, #37495, #37785, #38137, #38219)
• Wildcard IndicesPermissions don’t cover .security #36765

CCR

• Generalize search.remote settings to cluster.remote #33413
• Add ccr follow info api #37408 (issue: #37127)

Distributed

• Log messages from allocation commands #25955 (issues: #22821, #25325)

Features/ILM

• Add unfollow action #36970 (issue: #34648)

Features/Ingest

• Revert “Introduce a Hashing Processor (#31087)” #32178
• Add ingest-attachment support for per document indexed_chars limit #28977 (issue: #28942)

Features/Java High Level REST Client

• GraphClient for the high level REST client and associated tests #32366

Features/Monitoring

• Collect only display_name (for now) #35265 (issue: #8445)

Geo

• Integrate Lucene’s LatLonShape (BKD Backed GeoShapes) as default geo_shape indexing approach #36751 (issue: #35320)
• Integrate Lucene’s LatLonShape (BKD Backed GeoShapes) as default geo_shape indexing approach #35320 (issue: #32039)
• geotile_grid implementation #37842 (issue: #30240)
• Fork Lucene’s LatLonShape Classes to local lucene package #36794
• Integrate Lucene’s LatLonShape (BKD Backed GeoShapes) as default geo_shape indexing approach #36751 (issue: #35320)
• Integrate Lucene’s LatLonShape (BKD Backed GeoShapes) as default geo_shape indexing approach #35320 (issue: #32039)

Infra/Core

• Skip shard refreshes if shard is search idle #27500

Infra/Logging

• Logging: Unify log rotation for index/search slow log #27298

Infra/Plugins

• Reload secure settings for plugins #31383 (issue: #29135)

Infra/REST API

• Add an include_type_name option. #29453 (issue: #15613)

Java High Level REST Client

• Add rollup search #36334 (issue: #29827)

Java Low Level REST Client

• Make warning behavior pluggable per request #36345
• Add PreferHasAttributeNodeSelector #36005

Machine Learning

• Filter undefined job groups from update job calendar actions #30757
• Add delayed datacheck to the datafeed job runner #35387 (issue: #35131)
• Adds set_upgrade_mode API endpoint #37837

Mapping

• Add a feature_vector field. #31102 (issue: #27552)
• Expose Lucene’s FeatureField. #30618
• Make typeless APIs usable with indices whose type name is different from _doc #35790 (issue: #35190)
• Give precedence to index creation when mixing typed templates with typeless index creation and vice-versa. #37871 (issue: #37773)
• Add nanosecond field mapper #37755 (issues: #27330, #32601)

Ranking

• Add ranking evaluation API #27478 (issue: #19195)

Recovery

• Allow to trim all ops above a certain seq# with a term lower than X, … #31211 (issue: #10708)

SQL

• Add basic support for ST_AsWKT geo function #34205
• Add support for SYS GEOMETRY_COLUMNS #30496 (issue: #29872)
• Introduce HISTOGRAM grouping function #36510 (issue: #36509)
• DATABASE() and USER() system functions #35946 (issue: #35863)
• Introduce INTERVAL support #35521 (issue: #29990)
• Allow sorting of groups by aggregates #38042 (issue: #35118)
• Implement FIRST/LAST aggregate functions #37936 (issue: #35639)
• Introduce SQL DATE data type #37693 (issue: #37340)

Search

• Add “took” timing info to response for _msearch/template API #30961 (issue: #30957)
• Add allow_partial_search_results flag to search requests with default setting true #28440 (issue: #27435)
• Enable adaptive replica selection by default #26522 (issue: #24915)
• Add intervals query #36135 (issues: #29636, #32406)
• Added soft limit to open scroll contexts #25244 #36009 (issue: #25244)
• Introduce ability to minimize round-trips in CCS #37828 (issues: #32125, #37566)
• Add script filter to intervals #36776
• Add the ability to set the number of hits to track accurately #36357 (issue: #33028)
• Add a maximum search request size. #26423
• Make IntervalQuery available via the Query DSL #36135 (issue: #29636)

Security

• Switch internal security index to “.security-7” #39337 (issue: #39284)

Suggesters

• Serialize suggestion responses as named writeables #30284 (issue: #26585)

Enhancements

Aggregations

• Uses MergingDigest instead of AVLDigest in percentiles agg #28702 (issue: #19528)
• Added keyed response to pipeline percentile aggregations 22302 #36392 (issue: #22302)
• Enforce max_buckets limit only in the final reduction phase #36152 (issues: #32125, #35921)
• Histogram aggs: add empty buckets only in the final reduce step #35921
• Handles exists query in composite aggs #35758
• Added parent validation for auto date histogram #35670
• Add Composite to AggregationBuilders #38207 (issue: #38020)
• Allow nested fields in the composite aggregation #37178 (issue: #28611)
• Remove single shard optimization when suggesting shard_size #37041 (issue: #32125)
• Use List instead of priority queue for stable sorting in bucket sort aggregator #36748 (issue: #36322)
• Keys are compared in BucketSortPipelineAggregation so making key type… #36407

Allocation

• Fail start on obsolete indices documentation #37786 (issue: #27073)
• Fail start on invalid index metadata #37748 (issue: #27073)
• Fail start of non-data node if node has data #37347 (issue: #27073)

Analysis

• Allow word_delimiter_graph_filter to not adjust internal offsets #36699 (issues: #33710, #34741)
• Ensure TokenFilters only produce single tokens when parsing synonyms #34331 (issue: #34298)
• Allow word_delimiter_graph_filter to not adjust internal offsets #36699 (issues: #33710, #34741)

Audit

• Add “request.id” to file audit logs #35536
• Security Audit includes HTTP method for requests #37322 (issue: #29765)
• Add X-Forwarded-For to the logfile audit #36427

Authentication

• Invalidate Token API enhancements - HLRC #36362 (issue: #35388)
• Add DEBUG/TRACE logs for LDAP bind #36028
• Add Tests for findSamlRealm #35905
• Add realm information for Authenticate API #35648
• Formal support for “password_hash” in Put User #35242 (issue: #34729)
• Propagate auth result to listeners #36900 (issue: #30794)
• Reorder realms based on last success #36878
• Improve error message for 6.x style realm settings #36876 (issues: #30241, #36026)
• Change missing authn message to not mention tokens #36750
• Invalidate Token API enhancements - HLRC #36362 (issue: #35388)
• Enhance Invalidate Token API #35388 (issues: #34556, #35115)

Authorization

• Improve exact index matching performance #36017
• manage_token privilege for kibana_system #35751
• Grant .tasks access to kibana_system role #35573
• Add apm_user reserved role #38206
• Permission for restricted indices #37577 (issue: #34454)
• Remove kibana_user and kibana_dashboard_only_user index privileges #37441
• Create snapshot role #35820 (issue: #34454)

Build

• Sounds like typo in exception message #35458
• Allow set section in setup section of REST tests #34678

CCR

• Add time since last auto follow fetch to auto follow stats #36542 (issues: #33007, #35895)
• Clean followed leader index UUIDs in auto follow metadata #36408 (issue: #33007)
• Change AutofollowCoordinator to use wait_for_metadata_version #36264 (issues: #33007, #35895)
• Refactor AutoFollowCoordinator to track leader indices per remote cluster #36031 (issues: #33007, #35895)
• Concurrent file chunk fetching for CCR restore #38495
• Tighten mapping syncing in ccr remote restore #38071 (issues: #36879, #37887)
• Do not allow put mapping on follower #37675 (issue: #30086)
• Added ccr to xpack usage infrastructure #37256 (issue: #37221)
• FollowingEngine should fail with 403 if operation has no seqno assigned #37213
• Added auto_follow_exception.timestamp field to auto follow stats #36947
• Add time since last auto follow fetch to auto follow stats #36542 (issues: #33007, #35895)
• Reduce retention lease sync intervals #40302
• Renew retention leases while following #39335 (issues: #37165, #38718)
• Reduce refresh when lookup term in FollowingEngine #39184
• Integrate retention leases to recovery from remote #38829 (issue: #37165)
• Enable removal of retention leases #38751 (issue: #37165)
• Introduce forget follower API #39718 (issue: #37165)

Client

• Fixed required fields and paths list #39358

Cluster Coordination

• Remove timeout task after completing cluster state publication #40411
• Use default discovery implementation for single-node discovery #40036
• Do not log unsuccessful join attempt each time #39756

Core

• Override the JVM DNS cache policy #36570
• Replace usages of AtomicBoolean based block of code by the RunOnce class #35553 (issue: #35489)
• Added wait_for_metadata_version parameter to cluster state api. #35535
• Extract RunOnce into a dedicated class #35489
• Introduce elasticsearch-core jar #28191 (issue: #27933)
• Rename core module to server #28180 (issue: #27933)

CRUD

• Rename seq# powered optimistic concurrency control parameters to ifSeqNo/ifPrimaryTerm #36757 (issues: #10708, #36148)
• Expose Sequence Number based Optimistic Concurrency Control in the rest layer #36721 (issues: #10708, #36148)
• Add doc’s sequence number + primary term to GetResult and use it for updates #36680 (issues: #10708, #36148)
• Add seq no powered optimistic locking support to the index and delete transport actions #36619 (issues: #10708, #36148)
• Add Seq# based optimistic concurrency control to UpdateRequest #37872 (issues: #10708, #36148)
• Introduce ssl settings to reindex from remote #37527 (issues: #29755, #37287)
• Use Sequence number powered OCC for processing updates #37308 (issues: #10708, #36148)
• Document Seq No powered optimistic concurrency control #37284 (issues: #10708, #36148)
• Enable IPv6 URIs in reindex from remote #36874
• Rename seq# powered optimistic concurrency control parameters to ifSeqNo/ifPrimaryTerm #36757 (issues: #10708, #36148)
• Expose Sequence Number based Optimistic Concurrency Control in the rest layer #36721 (issues: #10708, #36148)
• Add doc’s sequence number + primary term to GetResult and use it for updates #36680 (issues: #10708, #36148)
• Add seq no powered optimistic locking support to the index and delete transport actions #36619 (issues: #10708, #36148)
• Set acking timeout to 0 on dynamic mapping update #31140 (issues: #30672, #30844)

Discovery-Plugins

• Adds connect and read timeouts to discovery-gce #28193 (issue: #24313)

Distributed

• [Close Index API] Mark shard copy as stale if needed during shard verification #36755
• [Close Index API] Refactor MetadataIndexStateService #36354 (issue: #36249)
• [Close Index API] Add TransportShardCloseAction for pre-closing verifications #36249
• TransportResyncReplicationAction should not honour blocks #35795 (issues: #35332, #35597)
• Expose all permits acquisition in IndexShard and TransportReplicationAction #35540 (issue: #33888)
• [RCI] Check blocks while having index shard permit in TransportReplicationAction #35332 (issue: #33888)
• Recover retention leases during peer recovery #38435 (issue: #37165)
• Lift retention lease expiration to index shard #38380 (issues: #37165, #37963, #38070)
• Introduce retention lease background sync #38262 (issue: #37165)
• Allow shards of closed indices to be replicated as regular shards #38024 (issue: #33888)
• Expose retention leases in shard stats #37991 (issue: #37165)
• Introduce retention leases versioning #37951 (issue: #37165)
• Soft-deletes policy should always fetch latest leases #37940 (issues: #37165, #37375)
• Sync retention leases on expiration #37902 (issue: #37165)
• Ignore shard started requests when primary term does not match #37899 (issue: #33888)
• Move update and delete by query to use seq# for optimistic concurrency control #37857 (issues: #10708, #36148, #37639)
• Introduce retention lease serialization #37447 (issues: #37165, #37398)
• Add run under primary permit method #37440 (issue: #37398)
• Introduce retention lease syncing #37398 (issue: #37165)
• Introduce retention lease persistence #37375 (issue: #37165)
• Add validation for retention lease construction #37312 (issue: #37165)
• Introduce retention lease expiration #37195 (issue: #37165)
• Introduce shard history retention leases #37167 (issue: #37165)
• [Close Index API] Add unique UUID to ClusterBlock #36775
• [Close Index API] Mark shard copy as stale if needed during shard verification #36755
• [Close Index API] Propagate tasks ids between Freeze, Close and Verify Shard actions #36630
• Always initialize the global checkpoint #34381
• Introduce retention lease actions #38756 (issue: #37165)
• Add dedicated retention lease exceptions #38754 (issue: #37165)
• Copy retention leases when trim unsafe commits #37995 (issue: #37165)
• Allow retention lease operations under blocks #39089 (issues: #34648, #37165)
• Remove retention leases when unfollowing #39088 (issues: #34648, #37165)
• Introduce retention lease state file #39004 (issues: #37165, #38588, #39032)
• Enable soft-deletes by default for 7.0+ indices #38929 (issue: #36141)

Engine

• Remove versionType from translog #31945
• Do retry if primary fails on AsyncAfterWriteAction #31857 (issues: #31716, #31755)
• handle AsyncAfterWriteAction exception before listener is registered #31755 (issue: #31716)
• Use IndexWriter#flushNextBuffer to free memory #27753
• Remove pre 6.0.0 support from InternalEngine #27720
• Add sequence numbers based optimistic concurrency control support to Engine #36467 (issues: #10708, #36148)
• Require soft-deletes when access changes snapshot #36446
• Use delCount of SegmentInfos to calculate numDocs #36323
• Always configure soft-deletes field of IndexWriterConfig #36196 (issue: #36141)
• Enable soft-deletes by default on 7.0.0 or later #36141
• Always return false from refreshNeeded on ReadOnlyEngine #35837 (issue: #35785)
• Add a _freeze / _unfreeze API #35592 (issue: #34352)
• [RCI] Add IndexShardOperationPermits.asyncBlockOperations(ActionListener) #34902 (issue: #33888)
• Specialize pre-closing checks for engine implementations #38702
• Ensure that max seq # is equal to the global checkpoint when creating ReadOnlyEngines #37426
• Enable Bulk-Merge if all source remains #37269
• Rename setting to enable mmap #37070 (issue: #36668)
• Add hybridfs store type #36668
• Introduce time-based retention policy for soft-deletes #34943 (issue: #34908)
• Handle AsyncAfterWriteAction failure on primary in the same way as failures on replicas #31969 (issues: #31716, #31755)
• Explicitly advance max_seq_no before indexing #39473 (issue: #38879)
• Also mmap cfs files for hybridfs #38940 (issue: #36668)

Features/CAT APIs

• Expose search.throttled on _cat/indices #37073 (issue: #34352)

Features/Features

• Run Node deprecation checks locally (#38065) #38250 (issue: #38065)

Features/ILM

• Ensure ILM policies run safely on leader indices #38140 (issue: #34648)
• Skip Shrink when numberOfShards not changed #37953 (issue: #33275)
• Inject Unfollow before Rollover and Shrink #37625 (issue: #34648)
• Add set_priority action to ILM #37397 (issue: #36905)
• Add Freeze Action #36910 (issue: #34630)

Features/Indices APIs

• Add cluster-wide shard limit #32856 (issue: #20705)
• Remove RestGetAllAliasesAction #31308 (issue: #31129)
• Add rollover-creation-date setting to rolled over index #31144 (issue: #30887)
• add is-write-index flag to aliases #30942
• Make index and bulk APIs work without types. #29479
• Simplify deprecation issue levels #36326
• New mapping signature and mapping string source fixed. #37401

Features/Ingest

• Add ignore_missing property to foreach filter (#22147) #31578 (issue: #22147)
• Compile mustache template only if field includes {{‘ #37207 (issue: #37120)
• Move ingest-geoip default databases out of config #36949 (issue: #36898)
• Make the ingest-geoip databases even lazier to load #36679
• Updates the grok patterns to be consistent with the logstash #27181

Features/Java High Level REST Client

• HLRC API for _termvectors #32610 (issue: #27205)
• Fix strict setting exception handling #37247 (issue: #37090)
• Use nonblocking entity for requests #32249

Features/Monitoring

• Make Exporters Async #35765 (issue: #35743)
• Adding mapping for hostname field #37288
• Remove types from internal monitoring templates and bump to api 7 #39888 (issue: #38637)

Features/Stats

• Stats to record how often the ClusterState diff mechanism is used successfully #26973
• Add JVM dns cache expiration config to JvmInfo #36372

Features/Watcher

• Validate email adresses when storing a watch #34042 (issue: #33980)
• Move watcher to use seq# and primary term for concurrency control #37977 (issues: #10708, #37872)
• Use ILM for Watcher history deletion #37443 (issue: #32041)
• Add whitelist to HttpClient #36817 (issue: #29937)
• Remove the index type from internal watcher indexes #39761 (issue: #38637)

Geo

• Adds a name of the field to geopoint parsing errors #36529 (issue: #15965)
• Add support to ShapeBuilders for building Lucene geometry #35707 (issue: #35320)
• Add ST_WktToSQL function #35416 (issue: #29872)

Index APIs

• Add cluster-wide shard limit warnings #34021 (issues: #20705, #32856)

Infra/Circuit Breakers

• Have circuit breaker succeed on unknown mem usage #33125 (issue: #31767)
• Account for XContent overhead in in-flight breaker #31613
• Script Stats: Add compilation limit counter to stats #26387

Infra/Core

• Add RunOnce utility class that executes a Runnable exactly once #35484
• Improved IndexNotFoundException’s default error message #34649 (issue: #34628)
• fix a few versionAdded values in ElasticsearchExceptions #37877
• Add simple method to write collection of writeables #37448 (issue: #37398)
• Date/Time parsing: Use java time API instead of exception handling #37222
• [API] spelling: interruptible #37049 (issue: #37035)
• Enhancements to IndicesQueryCache. #39099 (issue: #37117)
• Change zone formatting for all printers #39568 (issue: #38471)

Infra/Logging

• Trim the JSON source in indexing slow logs #38081 (issue: #38080)
• Optimize warning header de-duplication #37725 (issues: #35754, #37530, #37597, #37622)
• Remove warn-date from warning headers #37622 (issues: #35754, #37530, #37597)
• Add some deprecation optimizations #37597 (issues: #35754, #37530)
• Only update response headers if we have a new one #37590 (issues: #35754, #37530)

Infra/Packaging

• Choose JVM options ergonomically #30684
• Add OS/architecture classifier to distributions #37881
• Change file descriptor limit to 65535 #37537 (issue: #35839)
• Exit batch files explictly using ERRORLEVEL #29583 (issue: #29582)
• Add no-jdk distributions #39882
• Allow AVX-512 on JDK 11+ #40828 (issue: #32138)

Infra/REST API

• Remove hand-coded XContent duplicate checks #34588 (issues: #22073, #22225, #22253)
• Add the include_type_name option to the search and document APIs. #29506 (issue: #15613)
• Validate op_type for _create #27483

Infra/Scripting

• Tests: Add support for custom contexts to mock scripts #34100
• Reflect factory signatures in painless classloader #34088
• Handle missing values in painless #32207 (issue: #29286)
• Add getZone to JodaCompatibleZonedDateTime #37084
• [Painless] Add boxed type to boxed type casts for method/return #36571

Infra/Packaging

• Use bundled JDK in Docker images #40238
• Upgrade bundled JDK and Docker images to JDK 12 #40229
• Bundle java in distributions #38013 (issue: #31845)

Infra/Settings

• Settings: Add keystore creation to add commands #26126
• Separate out validation of groups of settings #34184
• Provide a clearer error message on keystore add #39327 (issue: #39324)

Infra/Transport API

• Change BWC version for VerifyRepositoryResponse #30796 (issue: #30762)

Ingest

• Grok fix duplicate patterns JAVACLASS and JAVAFILE #35886
• Implement Drop Processor #32278 (issue: #23726)

Java High Level REST Client

• Add get users action #36332 (issue: #29827)
• Add delete template API #36320 (issue: #27205)
• Implement get-user-privileges API #36292
• Get Deprecation Info API #36279 (issue: #29827)
• Add support for Follow Stats API #36253 (issue: #33824)
• Add support for CCR Stats API #36213 (issue: #33824)
• Put Role #36209 (issue: #29827)
• Add index templates exist API #36132 (issue: #27205)
• Add support for CCR Get Auto Follow Pattern apis #36049 (issue: #33824)
• Add support for CCR Delete Auto Follow Pattern API #35981 (issue: #33824)
• Remove fromXContent from IndexUpgradeInfoResponse #35934
• Add delete expired data API #35906 (issue: #29827)
• Execute watch API #35868 (issue: #29827)
• Add ability to put user with a password hash #35844 (issue: #35242)
• Add ML find file structure API #35833 (issue: #29827)
• Add support for get roles API #35787 (issue: #29827)
• Added support for CCR Put Auto Follow Pattern API #35780 (issue: #33824)
• XPack ML info action #35777 (issue: #29827)
• ML Delete event from Calendar #35760 (issue: #29827)
• Add ML revert model snapshot API #35750 (issue: #29827)
• ML Get Calendar Events #35747 (issue: #29827)
• Add high-level REST client API for _freeze and _unfreeze #35723 (issue: #34352)
• Fix issue in equals impl for GlobalOperationPrivileges #35721
• ML Delete job from calendar #35713 (issue: #29827)
• ML Add Event To Calendar API #35704 (issue: #29827)
• Add ML update model snapshot API (#35537) #35694 (issue: #29827)
• Add support for CCR Unfollow API #35693 (issue: #33824)
• Clean up PutLicenseResponse #35689 (issue: #35547)
• Clean up StartBasicResponse #35688 (issue: #35547)
• Add support for put privileges API #35679
• ML Add Job to Calendar API #35666 (issue: #29827)
• Add support for CCR Resume Follow API #35638 (issue: #33824)
• Add support for get application privileges API #35556 (issue: #29827)
• Clean up XPackInfoResponse class and related tests #35547
• Add parameters to stopRollupJob API #35545 (issue: #34811)
• Add ML delete model snapshot API #35537 (issue: #29827)
• Add get watch API #35531 (issue: #29827)
• Add ML Update Filter API #35522 (issue: #29827)
• Add ml get filters api #35502 (issue: #29827)
• Add ML get model snapshots API #35487 (issue: #29827)
• Add “_has_privileges” API to Security Client #35479 (issue: #29827)
• Add Delete Privileges API to HLRC #35454 (issue: #29827)
• Add support for CCR Put Follow API #35409
• Add ML delete filter action #35382 (issue: #29827)
• Add delete user action #35294 (issue: #29827)
• HLRC for _mtermvectors #35266 (issues: #27205, #33447)
• Reindex API with wait_for_completion false #35202 (issue: #27205)
• Add watcher stats API #35185 (issue: #29827)
• HLRC support for getTask #35166 (issue: #27205)
• Add GetRollupIndexCaps API #35102 (issue: #29827)
• HLRC: migration api - upgrade #34898 (issue: #29827)
• Add stop rollup job support to HL REST Client #34702 (issue: #29827)
• Bulk Api support for global parameters #34528 (issue: #26026)
• Add delete rollup job support to HL REST Client #34066 (issue: #29827)
• Add support for get license basic/trial status API #33176 (issue: #29827)
• Add machine learning open job #32860 (issue: #29827)
• Add ML HLRC wrapper and put_job API call #32726
• Add Get Snapshots High Level REST API #31537 (issue: #27205)

Java Low Level REST Client

• On retry timeout add root exception #25576

License

• Require acknowledgement to start_trial license #30135 (issue: #30134)
• Handle malformed license signatures #37137 (issue: #35340)

Machine Learning

• Create the ML annotations index #36731 (issues: #26034, #33376)
• Split in batches and migrate all jobs and datafeeds #36716 (issue: #32905)
• Add cluster setting to enable/disable config migration #36700 (issue: #32905)
• Add audits when deprecation warnings occur with datafeed start #36233
• Add lazy parsing for DatafeedConfig:Aggs,Query #36117
• Add support for lazy nodes (#29991) #34538 (issue: #29991)
• Move ML Optimistic Concurrency Control to Seq No #38278 (issues: #10708, #36148)
• Add explanation so far to file structure finder exceptions #38191 (issue: #29821)
• Add reason field in JobTaskState #38029 (issue: #34431)
• Add _meta information to all ML indices #37964
• Add upgrade mode docs, hlrc, and fix bug #37942
• Tighten up use of aliases rather than concrete indices #37874
• Add support for single bucket aggs in Datafeeds #37544 (issue: #36838)
• Create the ML annotations index #36731 (issues: #26034, #33376)
• Merge the Jindex master feature branch #36702 (issue: #32905)
• Add cluster setting to enable/disable config migration #36700 (issue: #32905)
• Allow stop unassigned datafeed and relax unset upgrade mode wait #39034

Mapping

• Log document id when MapperParsingException occurs #37800 (issue: #37658)
• [API] spelling: unknown #37056 (issue: #37035)
• Make SourceToParse immutable #36971
• Use index-prefix fields for terms of length min_chars - 1 #36703
• Introduce a parameter suppress_types_warnings. #38923

Network

• Add cors support to NioHttpServerTransport #30827 (issue: #28898)
• Reintroduce mandatory http pipelining support #30820
• Make http pipelining support mandatory #30695 (issues: #28898, #29500)
• Add nio http server transport #29587 (issue: #28898)
• Add class for serializing message to bytes #29384 (issue: #28898)
• Selectors operate on channel contexts #28468 (issue: #27260)
• Unify nio read / write channel contexts #28160 (issue: #27260)
• Create nio-transport plugin for NioTransport #27949 (issue: #27260)
• Add elasticsearch-nio jar for base nio classes #27801 (issue: #27802)
• Unify transport settings naming #36623
• Add sni name to SSLEngine in netty transport #33144 (issue: #32517)
• Add cors support to NioHttpServerTransport #30827 (issue: #28898)
• Reintroduce mandatory http pipelining support #30820
• Make http pipelining support mandatory #30695 (issues: #28898, #29500)
• Add nio http server transport #29587 (issue: #28898)
• Selectors operate on channel contexts #28468 (issue: #27260)
• Unify nio read / write channel contexts #28160 (issue: #27260)
• Create nio-transport plugin for NioTransport #27949 (issue: #27260)
• Add elasticsearch-nio jar for base nio classes #27801 (issue: #27802)
• Add NioGroup for use in different transports #27737 (issue: #27260)
• Add read timeouts to http module #27713
• Implement byte array reusage in NioTransport #27696 (issue: #27563)
• Introduce resizable inbound byte buffer #27551 (issue: #27563)
• Decouple nio constructs from the tcp transport #27484 (issue: #27260)
• Remove manual tracking of registered channels #27445 (issue: #27260)
• Remove tcp profile from low level nio channel #27441 (issue: #27260)
• Decouple ChannelFactory from Tcp classes #27286 (issue: #27260)
• Enable TLSv1.3 by default for JDKs with support #38103 (issue: #32276)

Packaging

• Introduce Docker images build #36246
• Move creation of temporary directory to Java #36002 (issue: #31003)

Percolator

• Make the type parameter optional when percolating existing documents. #39987 (issue: #39963)
• Add support for selecting percolator query candidate matches containing geo_point based queries #26040

Plugins

• Plugin install: don’t print download progress in batch mode #36361

Ranking

• Add k parameter to PrecisionAtK metric #27569
• Vector field #33022 (issue: #31615)

Recovery

• SyncedFlushService.getShardRoutingTable() should use metadata to check for index existence #37691 (issue: #33888)
• Make prepare engine step of recovery source non-blocking #37573 (issue: #37174)
• Make recovery source send operations non-blocking #37503 (issue: #37458)
• Prepare to make send translog of recovery non-blocking #37458 (issue: #37291)
• Make finalize step of recovery source non-blocking #37388 (issue: #37291)
• Make recovery source partially non-blocking #37291 (issue: #36195)
• Do not mutate RecoveryResponse #37204 (issue: #37174)
• Don’t block on peer recovery on the target side #37076 (issue: #36195)
• Reduce recovery time with compress or secure transport #36981 (issue: #33844)
• Translog corruption marker #33415 (issue: #31389)
• Do not wait for advancement of checkpoint in recovery #39006 (issues: #38949, #39000)

Rollup

• Add non-X-Pack centric rollup endpoints #36383 (issues: #35958, #35962)
• Add more diagnostic stats to job #35471
• Add wait_for_completion option to StopRollupJob API #34811 (issue: #34574)
• Replace the TreeMap in the composite aggregation #36675

Recovery

• Exposed engine must include all operations below global checkpoint during rollback #36159 (issue: #32867)

Scripting

• Update joda compat methods to use compat class #36654
• [Painless] Add boxed type to boxed type casts for method/return #36571
• [Painless] Add def to boxed type casts #36506

Settings

• Add user-defined cluster metadata #33325 (issue: #33220)

Search

• Make limit on number of expanded fields configurable #35284 (issues: #26541, #34778)
• Search: Simply SingleFieldsVisitor #34052
• Don’t count hits via the collector if the hit count can be computed from index stats. #33701
• Limit the number of concurrent requests per node #31206 (issue: #31192)
• Default max concurrent search req. numNodes * 5 #31171 (issues: #30783, #30994)
• Change ScriptException status to 400 (bad request) #30861 (issue: #12315)
• Change default value to true for transpositions parameter of fuzzy query #26901
• Introducing “took” time (in ms) for _msearch #23767 (issue: #23131)
• Add copy constructor to SearchRequest #36641 (issue: #32125)
• Add raw sort values to SearchSortValues transport serialization #36617 (issue: #32125)
• Add sort and collapse info to SearchHits transport serialization #36555 (issue: #32125)
• Add default methods to DocValueFormat #36480
• Respect indices options on _msearch #35887
• Allow efficient can_match phases on frozen indices #35431 (issues: #34352, #34357)
• Add a new query type - ScriptScoreQuery #34533 (issues: #23850, #27588, #30303)
• Tie break on cluster alias when merging shard search failures #38715 (issue: #38672)
• Add finalReduce flag to SearchRequest #38104 (issues: #37000, #37838)
• Streamline skip_unavailable handling #37672 (issue: #32125)
• Expose sequence number and primary terms in search responses #37639
• Add support for merging multiple search responses into one #37566 (issue: #32125)
• Allow field types to optimize phrase prefix queries #37436 (issue: #31921)
• Add support for providing absolute start time to SearchRequest #37142 (issue: #32125)
• Ensure that local cluster alias is never treated as remote #37121 (issues: #32125, #36997)
• [API] spelling: cacheable #37047 (issue: #37035)
• Add ability to suggest shard_size on coord node rewrite #37017 (issues: #32125, #36997, #37000)
• Skip final reduction if SearchRequest holds a cluster alias #37000 (issues: #32125, #36997)
• Add support for local cluster alias to SearchRequest #36997 (issue: #32125)
• Use SearchRequest copy constructor in ExpandSearchPhase #36772 (issue: #36641)
• Add raw sort values to SearchSortValues transport serialization #36617 (issue: #32125)
• Avoid BytesRef’s copying in ScriptDocValues’s Strings #29581 (issue: #29567)

Security

• Make credentials mandatory when launching xpack/migrate #36197 (issues: #29847, #33972)
• Move CAS operations in TokenService to sequence numbers #38311 (issues: #10708, #37872)
• Cleanup construction of interceptors #38294
• Add passphrase support to elasticsearch-keystore #37472 (issue: #32691)
• Types removal security index template #39705 (issue: #38637)
• Types removal security index template #39542 (issue: #38637)

Snapshot/Restore

• 31608 Add S3 Setting to Force Path Type Access #34721 (issue: #31608)

• Allow Parallel Restore Operations #36397
• Repo Creation out of ClusterStateTask #36157 (issue: #9488)
• Add read-only repository verification #35731 (issue: #35703)
• RestoreService should update primary terms when restoring shards of existing indices #38177 (issue: #33888)
• Allow open indices to be restored #37733
• Create specific exception for when snapshots are in progress #37550 (issue: #37541)
• Make Atomic Blob Writes Mandatory #37168 (issues: #37011, #37066)
• Speed up HDFS Repository Writes #37069
• Implement Atomic Blob Writes for HDFS Repository #37066 (issue: #37011)
• [API] spelling: repositories #37053 (issue: #37035)
• Use CancellableThreads to Abort #35901 (issue: #21759)
• S3 client encryption #30513 (issues: #11128, #16843)
• Mark Deleted Snapshot Directories with Tombstones #40228 (issue: #39852)

Stats

• Handle OS pretty name on old OS without OS release #35453 (issue: #35440)

Store

• Add RemoveCorruptedShardDataCommand #32281 (issues: #31389, #32279)
• Add option to force load term dict into memory #39741

SQL

• Introduce support for NULL values #34573 (issue: #32079)
• Extend the ODBC metric by differentiating between 32 and 64bit platforms #36753 (issue: #36740)
• Fix wrong appliance of StackOverflow limit for IN #36724 (issue: #36592)
• Introduce NOW/CURRENT_TIMESTAMP function #36562 (issue: #36534)
• Move requests’ parameters to requests JSON body #36149 (issue: #35992)
• Make INTERVAL millis optional #36043 (issue: #36032)
• Implement data type verification for conditionals #35916 (issue: #35907)
• Implement GREATEST and LEAST functions #35879 (issue: #35878)
• Implement null safe equality operator <=> #35873 (issue: #35871)
• SYS COLUMNS returns ODBC specific schema #35870 (issue: #35376)
• Polish grammar for intervals #35853
• Add filtering to SYS TYPES #35852 (issue: #35342)
• Implement NULLIF(expr1, expr2) function #35826 (issue: #35818)
• Lock down JDBC driver #35798 (issue: #35437)
• Implement NVL(expr1, expr2) #35794 (issue: #35782)
• Implement ISNULL(expr1, expr2) #35793 (issue: #35781)
• Implement IFNULL variant of COALESCE #35762 (issue: #35749)
• XPack FeatureSet functionality #35725 (issue: #34821)
• Perform lazy evaluation of mismatched mappings #35676 (issues: #35659, #35675)
• Improve validation of unsupported fields #35675 (issue: #35673)
• Move internals from Joda to java.time #35649 (issue: #35633)
• Allow look-ahead resolution of aliases for WHERE clause #38450 (issue: #29983)
• Implement CURRENT_DATE #38175 (issue: #38160)
• Generate relevant error message when grouping functions are not used in GROUP BY #38017 (issue: #37952)
• Skip the nested and object field types in case of an ODBC request #37948 (issue: #37801)
• Add protocol tests and remove jdbc_type from drivers response #37516 (issues: #36635, #36882)
• Remove slightly used meta commands #37506 (issue: #37409)
• Describe aliases as views #37496 (issue: #37422)
• Make FULL non-reserved keyword in the grammar #37377 (issue: #37376)
• Use declared source for error messages #37161
• Improve error message when unable to translate to ES query DSL #37129 (issue: #37040)
• [API] spelling: subtract #37055 (issue: #37035)
• [API] spelling: similar #37054 (issue: #37035)
• [API] spelling: input #37048 (issue: #37035)
• Enhance message for PERCENTILE[_RANK] with field as 2nd arg #36933 (issue: #36903)
• Preserve original source for each expression #36912 (issue: #36894)
• Extend the ODBC metric by differentiating between 32 and 64bit platforms #36753 (issue: #36740)
• Fix wrong appliance of StackOverflow limit for IN #36724 (issue: #36592)
• Enhance checks for inexact fields #39427 (issue: #38501)
• Change the default precision for CURRENT_TIMESTAMP function #39391 (issue: #39288)
• Add “fuzziness” option to QUERY and MATCH function predicates #40529 (issue: #40495)
• Add “validate.properties” property to JDBC’s allowed list of settings #39050 (issue: #38068)

Suggesters

• Remove unused empty constructors from suggestions classes #37295
• [API] spelling: likelihood #37052 (issue: #37035)

Task Management

• Periodically try to reassign unassigned persistent tasks #36069 (issue: #35792)
• Only require task permissions #35667 (issue: #35573)
• Retry if task can’t be written #35054 (issue: #33764)

ZenDiscovery

• Introduce vote withdrawal #35446
• Add basic Zen1 transport-level BWC #35443
• Add diff-based publishing #35290
• Introduce auto_shrink_voting_configuration setting #35217
• Introduce transport API for cluster bootstrapping #34961
• Reconfigure cluster as its membership changes #34592 (issue: #33924)
• Fail fast on disconnects #34503
• Add storage-layer disruptions to CoordinatorTests #34347
• Add low-level bootstrap implementation #34345
• Gather votes from all nodes #34335
• Add Cluster State Applier #34257
• Add safety phase to CoordinatorTests #34241
• Integrate FollowerChecker with Coordinator #34075
• Integrate LeaderChecker with Coordinator #34049
• Trigger join when active master detected #34008
• Update PeerFinder term on term bump #33992
• Calculate optimal cluster configuration #33924
• Introduce FollowersChecker #33917
• Integrate publication pipeline into Coordinator #33771
• Add DisruptableMockTransport #33713
• Implement basic cluster formation #33668
• Introduce LeaderChecker #33024
• Add leader-side join handling logic #33013
• Add PeerFinder#onFoundPeersUpdated #32939
• Introduce PreVoteCollector #32847
• Introduce ElectionScheduler #32846
• Introduce ElectionScheduler #32709
• Add HandshakingTransportAddressConnector #32643 (issue: #32246)
• Add UnicastConfiguredHostsResolver #32642 (issue: #32246)
• Cluster state publication pipeline #32584 (issue: #32006)
• Introduce gossip-like discovery of master nodes #32246
• Add core coordination algorithm for cluster state publishing #32171 (issue: #32006)
• Add term and config to cluster state #32100 (issue: #32006)
• Add discovery types to cluster stats #36442
• Introduce zen2 discovery type #36298
• Persist cluster states the old way on non-master-eligible nodes #36247 (issue: #3)
• Storage layer WriteStateException propagation #36052
• Implement Tombstone REST APIs #36007
• Update default for USE_ZEN2 to true #35998
• Add warning if cluster fails to form fast enough #35993
• Allow Setting a List of Bootstrap Nodes to Wait for #35847
• VotingTombstone class #35832
• PersistedState interface implementation #35819
• Support rolling upgrades from Zen1 #35737
• Add lag detector #35685
• Move ClusterState fields to be persisted to ClusterState.Metadata #35625
• Introduce ClusterBootstrapService #35488
• Introduce vote withdrawal #35446
• Add basic Zen1 transport-level BWC #35443
• Add elasticsearch-node detach-cluster tool #37979
• Deprecate minimum_master_nodes #37868
• Step down as master when configured out of voting configuration #37802 (issue: #37712)
• Enforce cluster UUIDs #37775
• Bubble exceptions up in ClusterApplierService #37729
• Use m_m_nodes from Zen1 master for Zen2 bootstrap #37701
• Add tool elasticsearch-node unsafe-bootstrap #37696
• Report terms and version if cluster does not form #37473
• Bootstrap a Zen2 cluster once quorum is discovered #37463
• Zen2: Add join validation #37203
• Publish cluster states in chunks #36973

Bug fixes

Aggregations

• Fix InternalAutoDateHistogram reproducible failure #32723 (issue: #32215)
• fix MultiValuesSourceFieldConfig toXContent #36525 (issue: #36474)
• Cache the score of the parent document in the nested agg #36019 (issues: #34555, #35985)
• Correct implemented interface of ParsedReverseNested #35455 (issue: #35449)
• Handle IndexOrDocValuesQuery in composite aggregation #35392
• Don’t load global ordinals with the map execution_hint #37833 (issue: #37705)
• Issue #37303 - Invalid variance fix #37384 (issue: #37303)
• Skip sibling pipeline aggregators reduction during non-final reduce #40101 (issue: #40059)
• Extend nextDoc to delegate to the wrapped doc-value iterator for date_nanos #39176 (issue: #39107)
• Only create MatrixStatsResults on final reduction #38130 (issue: #37587)

Allocation

• Fix _host based require filters #38173
• ALLOC: Fail Stale Primary Alloc. Req. without Data #37226 (issue: #37098)

Analysis

• Close #26771: beider_morse phonetic encoder failure when languageset unspecified #26848 (issue: #26771)
• Fix PreConfiguredTokenFilters getSynonymFilter() implementations #38839 (issue: #38793)

Audit

• Fix origin.type for connection_* events #36410
• Fix IndexAuditTrail rolling restart on rollover edge #35988 (issue: #33867)
• Fix NPE in Logfile Audit Filter #38120 (issue: #38097)
• LoggingAuditTrail correctly handle ReplicatedWriteRequest #39925 (issue: #39555)

Authorization

• Empty GetAliases authorization fix #34444 (issue: #31952)

Authentication

• Fix kerberos setting registration #35986 (issues: #30241, #35942)
• Add support for Kerberos V5 Oid #35764 (issue: #34763)
• Enhance parsing of StatusCode in SAML Responses #38628
• Limit token expiry to 1 hour maximum #38244
• Fix expired token message in Exception header #37196
• Fix NPE in CachingUsernamePasswordRealm #36953 (issue: #36951)
• Allow non super users to create API keys #40028 (issue: #40029)
• Use consistent view of realms for authentication #38815 (issue: #30301)
• Correct authenticate response for API key #39684
• Fix security index auto-create and state recovery race #39582

Build

• Use explicit deps on test tasks for check #36325
• Fix jdbc jar pom to not include deps #36036 (issue: #32014)
• Fix official plugins list #35661 (issue: #35623)

CCR

• Fix follow stats API’s follower index filtering feature #36647
• AutoFollowCoordinator should tolerate that auto follow patterns may be removed #35945 (issue: #35937)
• Only auto follow indices when all primary shards have started #35814 (issue: #35480)
• Avoid NPE in follower stats when no tasks metadata #35802
• Fix the names of CCR stats endpoints in usage API #35438
• Prevent CCR recovery from missing documents #38237
• Fix file reading in ccr restore service #38117
• Correct argument names in update mapping/settings from leader #38063
• Ensure changes requests return the latest mapping version #37633
• Do not set fatal exception when shard follow task is stopped. #37603
• Add fatal_exception field for ccr stats in monitoring mapping #37563
• Do not add index event listener if CCR disabled #37432
• When removing an AutoFollower also mark it as removed. #37402 (issue: #36761)
• Make shard follow tasks more resilient for restarts #37239 (issue: #37231)
• Resume follow Api should not require a request body #37217 (issue: #37022)
• Report error if auto follower tries auto follow a leader index with soft deletes disabled #36886 (issue: #33007)
• Remote cluster license checker and no license info. #36837 (issue: #36815)
• Make CCR resilient against missing remote cluster connections #36682 (issues: #36255, #36667)
• AutoFollowCoordinator and follower index already created #36540 (issue: #33007)
• Safe publication of AutoFollowCoordinator #40153 (issue: #38560)
• Enable reading auto-follow patterns from x-content #40130 (issue: #40128)
• Stop auto-followers on shutdown #40124
• Protect against the leader index being removed #39351 (issue: #39308)
• Handle the fact that ShardStats instance may have no commit or seqno stats #38782 (issue: #38779)
• Fix LocalIndexFollowingIT#testRemoveRemoteConnection() test #38709 (issue: #38695)
• Fix shard follow task startup error handling #39053 (issue: #38779)
• Filter out upgraded version index settings when starting index following #38838 (issue: #38835)

Circuit Breakers

• Modify BigArrays to take name of circuit breaker #36461 (issue: #31435)

Core

• Fix CompositeBytesReference#slice to not throw AIOOBE with legal offsets. #35955 (issue: #35950)
• Suppress CachedTimeThread in hot threads output #35558 (issue: #23175)
• Upgrade to Joda 2.10.1 #35410 (issue: #33749)

CRUD

• Fix Reindex from remote query logic #36908
• Synchronize WriteReplicaResult callbacks #36770
• Cascading primary failure lead to MSU too low #40249
• Store Pending Deletions Fix #40345 (issue: #40249)
• ShardBulkAction ignore primary response on primary #38901

Cluster Coordination

• Fix node tool cleanup #39389
• Avoid serialising state if it was already serialised #39179
• Do not perform cleanup if Manifest write fails with dirty exception #40519 (issue: #39077)
• Cache compressed cluster state size #39827 (issue: #39806)
• Drop node if asymmetrically partitioned from master #39598
• Fixing the custom object serialization bug in diffable utils. #39544
• Clean GatewayAllocator when stepping down as master #38885

Distributed

• Combine the execution of an exclusive replica operation with primary term update #36116 (issue: #35850)
• ActiveShardCount should not fail when closing the index #35936
• TransportVerifyShardBeforeCloseAction should force a flush #38401 (issues: #33888, #37961)
• Fix limit on retaining sequence number #37992 (issue: #37165)
• Close Index API should force a flush if a sync is needed #37961 (issues: #33888, #37426)
• Force Refresh Listeners when Acquiring all Operation Permits #36835
• Replaced the word shards with replicas in an error message. (#36234) #36275 (issue: #36234)
• Ignore waitForActiveShards when syncing leases #39224 (issue: #39089)
• Fix synchronization in LocalCheckpointTracker#contains #38755 (issues: #33871, #38633)
• Enforce retention leases require soft deletes #39922 (issue: #39914)
• Treat TransportService stopped error as node is closing #39800 (issue: #39584)
• Use cause to determine if node with primary is closing #39723 (issue: #39584)
• Don’t ack if unable to remove failing replica #39584 (issue: #39467)
• Fix NPE on Stale Index in IndicesService #38891 (issue: #38845)

Engine

• Set Lucene version upon index creation. #36038 (issue: #33826)
• Wrap can_match reader with ElasticsearchDirectoryReader #35857
• Copy checkpoint atomically when rolling generation #35407
• Subclass NIOFSDirectory instead of using FileSwitchDirectory #37140 (issues: #36668, #37111)
• Bubble up exception when processing NoOp #39338 (issue: #38898)
• ReadOnlyEngine should update translog recovery state information #39238
• Advance max_seq_no before add operation to Lucene #38879 (issue: #31629)

Features/Features

• Only count some fields types for deprecation check #40166
• Deprecation check for indices with very large numbers of fields #39869 (issue: #39851)

Features/ILM

• Preserve ILM operation mode when creating new lifecycles #38134 (issues: #38229, #38230)
• Retry ILM steps that fail due to SnapshotInProgressException #37624 (issues: #37541, #37552)
• Remove indexing_complete when removing policy #36620
• Handle failure to release retention leases in ILM #39281 (issue: #39181)
• Correct ILM metadata minimum compatibility version #40569 (issue: #40565)
• Handle null retention leases in WaitForNoFollowersStep #40477
• Allow ILM to stop if indices have nonexistent policies #40820 (issue: #40824)

Features/Indices APIs

• Validate top-level keys for create index request (#23755) #23869 (issue: #23755)
• Reject delete index requests with a body #37501 (issue: #8217)
• Fix duplicate phrase in shrink/split error message #36734 (issue: #36729)
• Get Aliases with wildcard exclusion expression #34230 (issues: #33518, #33805, #34144)

Features/Ingest

• Fix Deprecation Warning in Script Proc. #32407
• Support unknown fields in ingest pipeline map configuration #38352 (issue: #36938)
• Ingest node - user_agent, move device parsing to an object #38115 (issues: #37329, #38094)
• Fix on_failure with Drop processor #36686 (issue: #36151)
• Support default pipelines + bulk upserts #36618 (issue: #36219)
• Ingest ingest then create index #39607 (issues: #32758, #32786, #36545)

Features/Java High Level REST Client

• Drop extra level from user parser #34932
• Update IndexTemplateMetadata to allow unknown fields #38448 (issue: #36938)
• if_seq_no and if_primary_term parameters aren’t wired correctly in REST Client’s CRUD API #38411
• Update Rollup Caps to allow unknown fields #38339 (issue: #36938)
• Fix ILM explain response to allow unknown fields #38054 (issue: #36938)
• Fix ILM status to allow unknown fields #38043 (issue: #36938)
• Fix ILM Lifecycle Policy to allow unknown fields #38041 (issue: #36938)
• Update authenticate to allow unknown fields #37713 (issue: #36938)
• Update verify repository to allow unknown fields #37619 (issue: #36938)
• Update get users to allow unknown fields #37593 (issue: #36938)
• Update Execute Watch to allow unknown fields #37498 (issue: #36938)
• Update Put Watch to allow unknown fields #37494 (issue: #36938)
• Update Delete Watch to allow unknown fields #37435 (issue: #36938)
• Fix rest reindex test for IPv4 addresses #37310
• Fix weighted_avg parser not found for RestHighLevelClient #37027 (issue: #36861)

Features/Java Low Level REST Client

• Remove I/O pool blocking sniffing call from onFailure callback, add some logic around host exclusion #27985 (issue: #27984)
• Fix potential IllegalCapacityException in LLRC when selecting nodes #37821

Features/Monitoring

• Allow built-in monitoring_user role to call GET _xpack API #38060 (issue: #37970)
• Don’t emit deprecation warnings on calls to the monitoring bulk API. #39805 (issue: #39336)

Features/Watcher

• Ignore system locale/timezone in croneval CLI tool #33215
• Support merge nested Map in list for JIRA configurations #37634 (issue: #30068)
• Watcher accounts constructed lazily #36656
• Ensures watch definitions are valid json #30692 (issue: #29746)
• Use non-ILM template setting up watch history template & ILM disabled #39325 (issue: #38805)
• Only flush Watcher’s bulk processor if Watcher is enabled #38803 (issue: #38798)
• Fix Watcher stats class cast exception #39821 (issue: #39780)
• Use any index specified by .watches for Watcher #39541 (issue: #39478)
• Resolve concurrency with watcher trigger service #39092 (issue: #39087)
• Metric on watcher stats is a list not an enum #39114

Geo

• Test GeoShapeQueryTests#testPointsOnly fails #27454
• More robust handling of ignore_malformed in geoshape parsing #35603 (issues: #34047, #34498)
• Better handling of malformed geo_points #35554 (issue: #35419)
• Enables coerce support in WKT polygon parser #35414 (issue: #35059)
• Fix GeoHash PrefixTree BWC #38584 (issue: #38494)
• Do not normalize the longitude with value -180 for Lucene shapes #37299 (issue: #37297)
• Geo Point parse error fix #40447 (issue: #17617)

Highlighting

• Bug fix for AnnotatedTextHighlighter - port of 39525 #39750 (issue: #39525)

Infra/Core

• Ensure shard is refreshed once it’s inactive #27559 (issue: #27500)
• Bubble-up exceptions from scheduler #38317 (issue: #38014)
• Revert back to joda’s multi date formatters #36814 (issues: #36447, #36602)
• Propagate Errors in executors to uncaught exception handler #36137 (issue: #28667)
• Correct name of basic_date_time_no_millis #39367
• Allow single digit milliseconds in strict date parsing #40676 (issue: #40403)
• Parse composite patterns using ClassicFormat.parseObject #40100 (issue: #39916)
• Bat scripts to work with JAVA_HOME with parantheses #39712 (issues: #30606, #33405, #38578, #38624)
• Change licence expiration date pattern #39681 (issue: #39136)
• Fix DateFormatters.parseMillis when no timezone is given #39100 (issue: #39067)
• Don’t close caches while there might still be in-flight requests. #38958 (issue: #37117)
• Allow single digit milliseconds in strict date parsing #40676 (issue: #40403)

Infra/Packaging

• Remove NOREPLACE for /etc/elasticsearch in rpm and deb #37839
• Packaging: Update marker used to allow ELASTIC_PASSWORD #37243 (issue: #37240)
• Remove permission editing in postinst #37242 (issue: #37143)
• Some elasticsearch-cli tools could not be run not from ES_HOME #39937
• Obsolete pre 7.0 noarch package in rpm #39472 (issue: #39414)
• Suppress error message when /proc/sys/vm/max_map_count is not exists. #35933
• Use TAR instead of DOCKER build type before 6.7.0 #40723 (issues: #39378, #40511)
• Source additional files correctly in elasticsearch-cli #40890 (issue: #40889)

Infra/REST API

• Reject all requests that have an unconsumed body #37504 (issues: #30792, #37501, #8217)
• Fix #38623 remove xpack namespace REST API #38625
• Remove the “xpack” namespace from the REST API #38623
• Update spec files that erroneously documented parts as optional #39122
• ilm.explain_lifecycle documents human again #39113
• Index on rollup.rollup_search.json is a list #39097

Infra/Scripting

• Fix Painless void return bug #38046
• Correct bug in ScriptDocValues #40488

Infra/Settings

• Change format how settings represent lists / array #26723
• Fix setting by time unit #37192
• Fix handling of fractional byte size value settings #37172
• Fix handling of fractional time value settings #37171

Infra/Transport API

• Remove version read/write logic in Verify Response #30879 (issue: #30807)
• Enable muted Repository test #30875 (issue: #30807)
• Bad regex in CORS settings should throw a nicer error #29108

Index APIs

• Fix duplicate phrase in shrink/split error message #36734 (issue: #36729)
• Raise a 404 exception when document source is not found (#33384) #34083 (issue: #33384)

Ingest

• Fix on_failure with Drop processor #36686 (issue: #36151)
• Support default pipelines + bulk upserts #36618 (issue: #36219)
• Support default pipeline through an alias #36231 (issue: #35817)

License

• Update versions for start_trial after backport #30218 (issue: #30135)
• Do not serialize basic license exp in x-pack info #30848
• Update versions for start_trial after backport #30218 (issue: #30135)

Machine Learning

• Interrupt Grok in file structure finder timeout #36588
• Prevent stack overflow while copying ML jobs and datafeeds #36370 (issue: #36360)
• Adjust file structure finder parser config #35935
• Fix find_file_structure NPE with should_trim_fields #35465 (issue: #35462)
• Prevent notifications being created on deletion of a non existent job #35337 (issues: #34058, #35336)
• Clear Job#finished_time when it is opened (#32605) #32755
• Fix thread leak when waiting for job flush (#32196) #32541 (issue: #32196)
• Fix CPoissonMeanConjugate sampling error. #335
• Report index unavailable instead of waiting for lazy node #38423
• Fix error race condition on stop _all datafeeds and close _all jobs #38113 (issue: #37959)
• Update ML results mappings on process start #37706 (issue: #37607)
• Prevent submit after autodetect worker is stopped #37700 (issue: #37108)
• Fix ML datafeed CCS with wildcarded cluster name #37470 (issue: #36228)
• Update error message for process update #37363
• Wait for autodetect to be ready in the datafeed #37349 (issues: #36810, #37227)
• Stop datafeeds running when their jobs are stale #37227 (issue: #36810)
• Order GET job stats response by job id #36841 (issue: #36683)
• Make GetJobStats work with arbitrary wildcards and groups #36683 (issue: #34745)
• Fix datafeed skipping first bucket after lookback when aggs are … #39859 (issue: #39842)
• Refactoring lazy query and agg parsing #39776 (issue: #39528)
• Stop the ML memory tracker before closing node #39111 (issue: #37117)
• Scrolling datafeed should clear scroll contexts on error #40773 (issue: #40772)

Mapping

• Ensure that field aliases cannot be used in multi-fields. #32219
• Treat put-mapping calls with _doc as a top-level key as typed calls. #38032
• Correct deprec log in RestGetFieldMappingAction #37843 (issue: #37667)
• Restore a noop _all metadata field for 6x indices #37808 (issue: #37429)
• Make sure PutMappingRequest accepts content types other than JSON. #37720
• Make sure to use the resolved type in DocumentMapperService#extractMappings. #37451 (issue: #36811)
• Improve Precision for scaled_float #37169 (issue: #32570)
• Make sure to accept empty unnested mappings in create index requests. #37089
• Stop automatically nesting mappings in index creation requests. #36924
• Rewrite SourceToParse with resolved docType #36921 (issues: #35790, #36769)
• Optimise rejection of out-of-range long values #40325 (issues: #26137, #40323)
• Make sure to reject mappings with type _doc when include_type_name is false. #38270 (issue: #38266)

Network

• Adjust SSLDriver behavior for JDK11 changes #32145 (issues: #32122, #32144)
• Netty4SizeHeaderFrameDecoder error #31057
• Fix memory leak in http pipelining #30815 (issue: #30801)
• Fix issue with finishing handshake in ssl driver #30580
• Do not resolve addresses in remote connection info #36671 (issue: #35658)
• Always compress based on the settings #36522 (issue: #36399)
• http.publish_host Should Contain CNAME #32806 (issue: #22029)
• Adjust SSLDriver behavior for JDK11 changes #32145 (issues: #32122, #32144)
• Add TRACE, CONNECT, and PATCH http methods #31035 (issue: #31017)
• Transport client: Don’t validate node in handshake #30737 (issue: #30141)
• Fix issue with finishing handshake in ssl driver #30580
• Remove potential nio selector leak #27825
• Fix issue where the incorrect buffers are written #27695 (issue: #27551)
• Do not set SO_LINGER on server channels #26997
• Do not set SO_LINGER to 0 when not shutting down #26871 (issue: #26764)
• Release pipelined http responses on close #26226
• Reload SSL context on file change for LDAP #36937 (issues: #30509, #36923)
• Do not resolve addresses in remote connection info #36671 (issue: #35658)

Packaging

• Fix error message when package install fails due to missing Java #36077 (issue: #31845)
• Add missing entries to conffiles #35810 (issue: #35691)

Plugins

• Ensure that azure stream has socket privileges #28751 (issue: #28662)

Ranking

• QueryRescorer should keep the window size when rewriting #36836

Recovery

• Register ResyncTask.Status as a NamedWriteable #36610
• RecoveryMonitor#lastSeenAccessTime should be volatile #36781
• Create retention leases file during recovery #39359 (issue: #37165)
• Recover peers from translog, ignoring soft deletes #38904 (issue: #37165)
• Retain history for peer recovery using leases #38855
• Resync should not send operations without sequence number #40433

Rollup

• Fix rollup search statistics #36674
• Fix Rollup’s metadata parser #36791 (issue: #36726)
• Fix rollup search statistics #36674
• Remove timezone validation on rollup range queries #40647
• Rollup ignores time_zone on date histogram #40844

Scripting

• Properly support no-offset date formatting #36316 (issue: #36306)
• [Painless] Generate Bridge Methods #36097
• Fix serialization bug in painless execute api request #36075 (issue: #36050)
• Actually add joda time back to whitelist #35965 (issue: #35915)
• Add back joda to whitelist #35915 (issue: #35913)

Settings

• Correctly Identify Noop Updates #36560 (issue: #36496)

Search

• Ensure realtime _get and _termvectors don’t run on the network thread #33814 (issue: #27500)
• [bug] fuzziness custom auto #33462 (issue: #33454)
• Fix inner hits retrieval when stored fields are disabled (none) #33018 (issue: #32941)
• Set maxScore for empty TopDocs to Nan rather than 0 #32938
• Handle leniency for cross_fields type in multi_match query #27045 (issue: #23210)
• Raise IllegalArgumentException instead if query validation failed #26811 (issue: #26799)
• Inner hits fail to propagate doc-value format. #36310
• Fix custom AUTO issue with Fuzziness#toXContent #35807 (issue: #33462)
• Fix analyzed prefix query in query_string #35756 (issue: #31702)
• Fix problem with MatchNoDocsQuery in disjunction queries #35726 (issue: #34708)
• Fix phrase_slop in query_string query #35533 (issue: #35125)
• Add a More Like This query routing requirement check (#29678) #33974
• Look up connection using the right cluster alias when releasing contexts #38570
• Fix fetch source option in expand search phase #37908 (issue: #23829)
• Change rational to saturation in script_score #37766 (issue: #37714)
• Throw if two inner_hits have the same name #37645 (issue: #37584)
• Ensure either success or failure path for SearchOperationListener is called #37467 (issue: #37185)
• query_string should use indexed prefixes #36895
• Avoid duplicate types deprecation messages in search-related APIs. #36802
• Serialize top-level pipeline aggs as part of InternalAggregations #40177 (issues: #40059, #40101)
• CCS: Skip empty search hits when minimizing round-trips #40098 (issues: #32125, #40067)
• CCS: Disable minimizing round-trips when dfs is requested #40044 (issue: #32125)
• Fix Fuzziness#asDistance(String) #39643 (issue: #39614)
• Fix alias resolution runtime complexity. #40263 (issue: #40248)

Security

• Handle 6.4.0+ BWC for Application Privileges #32929
• Remove license state listeners on closeables #36308 (issues: #33328, #35627, #35628)
• Fix exit code for Security CLI tools #37956 (issue: #37841)
• Fix potential NPE in UsersTool #37660
• Remove dynamic objects from security index #40499 (issue: #35460)
• Fix libs:ssl-config project setup #39074
• Do not create the missing index when invoking getRole #39039

Snapshot/Restore

• Upgrade GCS Dependencies to 1.55.0 #36634 (issues: #35229, #35459)
• Improve Resilience SnapshotShardService #36113 (issue: #32265)
• Keep SnapshotsInProgress State in Sync with Routing Table #35710
• Ensure that gcs client creation is privileged #25938 (issue: #25932)
• Make calls to CloudBlobContainer#exists privileged #25937 (issue: #25931)
• Fix Concurrent Snapshot Ending And Stabilize Snapshot Finalization #38368 (issue: #38226)
• Fix Two Races that Lead to Stuck Snapshots #37686 (issues: #32265, #32348)
• Fix Race in Concurrent Snapshot Delete and Create #37612 (issue: #37581)
• Streamline S3 Repository- and Client-Settings #37393
• Blob store compression fix #39073

SQL

• Fix translation of LIKE/RLIKE keywords #36672 (issues: #36039, #36584)
• Scripting support for casting functions CAST and CONVERT #36640 (issue: #36061)
• Fix translation to painless for conditionals #36636 (issue: #36631)
• Concat should be always not nullable #36601 (issue: #36169)
• Fix MOD() for long and integer arguments #36599 (issue: #36364)
• Fix issue with complex HAVING and GROUP BY ordinal #36594 (issue: #36059)
• Be lenient for tests involving comparison to H2 but strict for csv spec tests #36498 (issue: #36483)
• Non ISO 8601 versions of DAY_OF_WEEK and WEEK_OF_YEAR functions #36358 (issue: #36263)
• Do not ignore all fields whose names start with underscore #36214 (issue: #36206)
• Fix issue with wrong data type for scripted Grouping keys #35969 (issue: #35662)
• Fix translation of math functions to painless #35910 (issue: #35654)
• Fix jdbc jar to include deps #35602
• Fix query translation for scripted queries #35408 (issue: #35232)
• Clear the cursor if nested inner hits are enough to fulfill the query required limits #35398 (issue: #35176)
• Introduce IsNull node to simplify expressions #35206 (issues: #34876, #35171)
• The SSL default configuration shouldn’t override the https protocol if used #34635 (issue: #33817)
• Minor fix for javadoc #32573 (issue: #32553)
• Prevent grouping over grouping functions #38649 (issue: #38308)
• Relax StackOverflow circuit breaker for constants #38572 (issue: #38571)
• Fix issue with IN not resolving to underlying keyword field #38440 (issue: #38424)
• Change the Intervals milliseconds precision to 3 digits #38297 (issue: #37423)
• Fix esType for DATETIME/DATE and INTERVALS #38179 (issue: #38051)
• Added SSL configuration options tests #37875 (issue: #37711)
• Fix casting from date to numeric type to use millis #37869 (issue: #37655)
• Fix BasicFormatter NPE #37804
• Return Intervals in SQL format for CLI #37602 (issues: #29970, #36186, #36432)
• Fix object extraction from sources #37502 (issue: #37364)
• Fix issue with field names containing “.” #37364 (issue: #37128)
• Fix bug regarding alias fields with dots #37279 (issue: #37224)
• Proper handling of COUNT(field_name) and COUNT(DISTINCT field_name) #37254 (issue: #30285)
• Fix COUNT DISTINCT filtering #37176 (issue: #37086)
• Fix issue with wrong NULL optimization #37124 (issue: #35872)
• Fix issue with complex expression as args of PERCENTILE/_RANK #37102 (issue: #37099)
• Handle the bwc Joda ZonedDateTime scripting class in Painless #37024 (issue: #37023)
• Fix bug regarding histograms usage in scripting #36866
• Fix issue with always false filter involving functions #36830 (issue: #35980)
• Protocol returns ISO 8601 String formatted dates instead of Long for JDBC/ODBC requests #36800 (issue: #36756)
• Enhance Verifier to prevent aggregate or grouping functions from #36799 (issue: #36798)
• Fix translation of LIKE/RLIKE keywords #36672 (issues: #36039, #36584)
• Scripting support for casting functions CAST and CONVERT #36640 (issue: #36061)
• Concat should be always not nullable #36601 (issue: #36169)
• Fix issue with complex HAVING and GROUP BY ordinal #36594 (issue: #36059)
• Add missing handling of IP field in JDBC #40384 (issue: #40358)
• Fix metric aggs on date/time to not return double #40377 (issues: #39492, #40376)
• CAST supports both SQL and ES types #40365 (issue: #40282)
• Fix RLIKE bug and improve testing for RLIKE statement #40354 (issues: #34609, #39931)
• Unwrap the first value in an array in case of array leniency #40318 (issue: #40296)
• Preserve original source for cast/convert function #40271 (issue: #40239)
• Fix LIKE function equality by considering its pattern as well #40260 (issue: #39931)
• Fix issue with optimization on queries with ORDER BY/LIMIT #40256 (issue: #40211)
• Rewrite ROUND and TRUNCATE functions with a different optional parameter handling method #40242 (issue: #40001)
• Fix issue with getting DATE type in JDBC #40207
• Fix issue with date columns returned always in UTC #40163 (issue: #40152)
• Add multi_value_field_leniency inside FieldHitExtractor #40113 (issue: #39700)
• Fix incorrect ordering of groupings (GROUP BY) based on orderings (ORDER BY) #40087 (issue: #39956)
• Fix bug with JDBC timezone setting and DATE type #39978 (issue: #39915)
• Use underlying exact field for LIKE/RLIKE #39443 (issue: #39442)
• Fix display size for DATE/DATETIME #40669
• Have LIKE/RLIKE use wildcard and regexp queries #40628 (issue: #40557)
• Fix getTime() methods in JDBC #40484
• SYS TABLES: enumerate tables of requested types #40535 (issue: #40348)
• Passing an input to the CLI “freezes” the CLI after displaying an error message #40164 (issue: #40557)
• Wrap ZonedDateTime parameters inside scripts #39911 (issue: #39877)
• ConstantProcessor can now handle NamedWriteable #39876 (issue: #39875)
• Extend the multi dot field notation extraction to lists of values #39823 (issue: #39738)
• Values in datetime script aggs should be treated as long #39773 (issue: #37042)
• Don’t allow inexact fields for MIN/MAX #39563 (issue: #39427)
• Fix merging of incompatible multi-fields #39560 (issue: #39547)
• Fix COUNT DISTINCT column name #39537 (issue: #39511)
• Enable accurate hit tracking on demand #39527 (issue: #37971)
• Ignore UNSUPPORTED fields for JDBC and ODBC modes in SYS COLUMNS #39518 (issue: #39471)
• Enforce JDBC driver - ES server version parity #38972 (issue: #38775)
• Fall back to using the field name for column label #38842 (issue: #38831)

Suggesters

• Fix duplicate removal when merging completion suggestions #36996 (issue: #35836)

Task Management

• Un-assign persistent tasks as nodes exit the cluster #37656

Watcher

• Watcher accounts constructed lazily #36656
• Only trigger a watch if new or schedule/changed #35908
• Fix Watcher NotificationService’s secure settings #35610 (issue: #35378)
• Fix integration tests to ensure correct start/stop of Watcher #35271 (issues: #29877, #30705, #33291, #34448, #34462)

ZenDiscovery

• Remove duplicate discovered peers #35505
• Respect the no_master_block setting #36478
• Cancel GetDiscoveredNodesAction when bootstrapped #36423 (issues: #36380, #36381)
• Only elect master-eligible nodes #35996
• Remove duplicate discovered peers #35505
• Fix size of rolling-upgrade bootstrap config #38031
• Always return metadata version if metadata is requested #37674
• Elect freshest master in upgrade #37122 (issue: #40)
• Fix cluster state persistence for single-node discovery #36825

Regressions

Infra/Core

• Restore date aggregation performance in UTC case #38221 (issue: #37826)
• Speed up converting of temporal accessor to zoned date time #37915 (issue: #37826)

Mapping

• Performance fix. Reduce deprecation calls for the same bulk request #37415 (issue: #37411)

Scripting

• Use Number as a return value for BucketAggregationScript #35653 (issue: #35351)

Upgrades

Discovery-Plugins

• Bump jackson-databind version for AWS SDK #39183

Engine

• Upgrade to lucene-8.0.0-snapshot-83f9835. #37668
• Upgrade to Lucene 8.0.0-snapshot-ff9509a8df #39350
• Upgrade to Lucene 8.0.0 #39992 (issue: #39640)

Geo

• Upgrade JTS to 1.14.0 #29141 (issue: #29122)

Ingest

• Update geolite2 database in ingest geoip plugin #33840
• Bump jackson-databind version for ingest-geoip #39182

Infra/Core

• Upgrade to a Lucene 8 snapshot #33310 (issues: #32899, #33028, #33309)

Security

• Upgrade the bouncycastle dependency to 1.61 #40017 (issue: #40011)

Search

• Upgrade to Lucene 8.0.0 GA #39992 (issue: #39640)

Snapshot/Restore

• plugins/repository-gcs: Update google-cloud-storage/core to 1.59.0 #39748 (issue: #39366)

Network

• Fix Netty Leaks by upgrading to 4.1.28 #32511 (issue: #32487)
• Upgrade Netty 4.3.32.Final #36102 (issue: #35360)

Machine Learning

• No need to add state doc mapping on job open in 7.x #37759