Elasticsearch version 7.0.0-beta1

Also see Breaking changes in 7.0.

Breaking changes

Audit

Authentication

  • Remove bwc logic for token invalidation #36893 (issue: #36727)

Authorization

  • Remove implicit index monitor privilege #37774

CCR

  • Follow stats api should return a 404 when requesting stats for a non existing index #37220 (issue: #37021)

CRUD

  • Remove support for internal versioning for concurrency control #38254 (issue: #1078)

Features/Ingest

Features/Java Low Level REST Client

  • Drop support for the low-level REST client on JDK 7 #38540 (issue: #29607)

Features/Watcher

  • Remove Watcher Account “unsecure” settings #36736 (issue: #36403)

Infra/Logging

Infra/Packaging

  • Package ingest-user-agent as a module #36956
  • Package ingest-geoip as a module #36898

Machine Learning

Mapping

  • Make sure to reject mappings with type _doc when include_type_name is false. #38270 (issue: #38266)
  • Update the default for include_type_name to false. #37285
  • Support include_type_name in RestGetIndicesAction #37149

Network

  • Remove TLS 1.0 as a default SSL protocol #37512 (issue: #36021)
  • Security: remove SSL settings fallback #36846 (issue: #29797)

Ranking

  • Forbid negative field boosts in analyzed queries #37930 (issue: #33309)

Search

Security

  • Remove heuristics that enable security on trial licenses #38075 (issue: #38009)

ZenDiscovery

  • Remove DiscoveryPlugin#getDiscoveryTypes #38414 (issue: #38410)

Breaking Java changes

Features/Java Low Level REST Client

Infra/Core

Deprecations

Aggregations

Analysis

  • [Analysis] Deprecate Standard Html Strip Analyzer in master #26719 (issue: #4704)

Audit

Features/Indices APIs

  • Reject setting index.optimize_auto_generated_id after version 7.0.0 #28895 (issue: #27600)

Features/Ingest

  • Deprecate _type in simulate pipeline requests #37949 (issue: #37731)

Features/Java High Level REST Client

Features/Watcher

  • Deprecate xpack.watcher.history.cleaner_service.enabled #37782 (issue: #32041)
  • deprecate types for watcher #37594 (issue: #35190)

Infra/Core

  • Core: Deprecate negative epoch timestamps #36793
  • Core: Deprecate use of scientific notation in epoch time parsing #36691

Infra/Scripting

  • Add types deprecation to script contexts #37554
  • Deprecate _type from LeafDocLookup #37491
  • Scripting: Remove deprecated params.ctx #36848 (issue: #34059)

Machine Learning

  • Adding ml_settings entry to HLRC and Docs for deprecation_info #38118
  • [ML] Datafeed deprecation checks #38026 (issue: #37932)
  • [ML] Remove “8” prefixes from file structure finder timestamp formats #38016
  • [ML] Adjust structure finder for Joda to Java time migration #37306
  • [ML] Resolve 7.0.0 TODOs in ML code #36842 (issue: #29963)

Mapping

  • Deprecate types in rollover index API #38039 (issue: #35190)
  • Deprecate types in get field mapping API #37667 (issue: #35190)
  • Deprecate types in the put mapping API. #37280 (issues: #29453, #37285)
  • Support include_type_name in the field mapping and index template APIs. #37210
  • Deprecate types in create index requests. #37134 (issues: #29453, #37285)
  • Deprecate use of the _type field in aggregations. #37131 (issue: #36802)
  • Deprecate reference to _type in lookup queries #37016 (issue: #35190)
  • Deprecate the document create endpoint. #36863
  • Deprecate types in index API #36575 (issues: #35190, #35790)
  • Deprecate types in update APIs #36225

Search

  • Deprecate use of type in reindex request body #36823
  • Add typless endpoints for get_source and exist_source #36426

New features

Authentication

  • Add support for API keys to access Elasticsearch #38291 (issue: #34383)
  • OIDC realm authentication flows #37787
  • [WIP] OIDC Realm JWT+JWS related functionality #37272 (issues: #35339, #37009)
  • OpenID Connect Realm base functionality #37009 (issue: #35339)

Authorization

CCR

Features/ILM

Geo

  • geotile_grid implementation #37842 (issue: #30240)
  • [GEO] Fork Lucene’s LatLonShape Classes to local lucene package #36794
  • [Geo] Integrate Lucene’s LatLonShape (BKD Backed GeoShapes) as default geo_shape indexing approach #36751 (issue: #35320)
  • [Geo] Integrate Lucene’s LatLonShape (BKD Backed GeoShapes) as default geo_shape indexing approach #35320 (issue: #32039)

Machine Learning

  • ML: Adds set_upgrade_mode API endpoint #37837

Mapping

  • Give precedence to index creation when mixing typed templates with typeless index creation and vice-versa. #37871 (issue: #37773)
  • Add nanosecond field mapper #37755 (issues: #27330, #32601)

SQL

  • SQL: Allow sorting of groups by aggregates #38042 (issue: #35118)
  • SQL: Implement FIRST/LAST aggregate functions #37936 (issue: #35639)
  • SQL: Introduce SQL DATE data type #37693 (issue: #37340)

Search

  • Introduce ability to minimize round-trips in CCS #37828 (issues: #32125, #37566)
  • Add script filter to intervals #36776
  • Add the ability to set the number of hits to track accurately #36357 (issue: #33028)
  • Add a maximum search request size. #26423

Enhancements

Aggregations

  • Add Composite to AggregationBuilders #38207 (issue: #38020)
  • Allow nested fields in the composite aggregation #37178 (issue: #28611)
  • Remove single shard optimization when suggesting shard_size #37041 (issue: #32125)
  • Use List instead of priority queue for stable sorting in bucket sort aggregator #36748 (issue: #36322)
  • Keys are compared in BucketSortPipelineAggregation so making key type… #36407

Allocation

  • Fail start on obsolete indices documentation #37786 (issue: #27073)
  • Fail start on invalid index metadata #37748 (issue: #27073)
  • Fail start of non-data node if node has data #37347 (issue: #27073)

Analysis

  • Allow word_delimiter_graph_filter to not adjust internal offsets #36699 (issues: #33710, #34741)

Audit

  • Security Audit includes HTTP method for requests #37322 (issue: #29765)
  • Add X-Forwarded-For to the logfile audit #36427

Authentication

  • Security: propagate auth result to listeners #36900 (issue: #30794)
  • Security: reorder realms based on last success #36878
  • Improve error message for 6.x style realm settings #36876 (issues: #30241, #36026)
  • Change missing authn message to not mention tokens #36750
  • Invalidate Token API enhancements - HLRC #36362 (issue: #35388)
  • Enhance Invalidate Token API #35388 (issues: #34556, #35115)

Authorization

  • Add apm_user reserved role #38206
  • Permission for restricted indices #37577 (issue: #34454)
  • Remove kibana_user and kibana_dashboard_only_user index privileges #37441
  • Create snapshot role #35820 (issue: #34454)

CCR

  • Concurrent file chunk fetching for CCR restore #38495
  • Tighten mapping syncing in ccr remote restore #38071 (issues: #36879, #37887)
  • Do not allow put mapping on follower #37675 (issue: #30086)
  • Added ccr to xpack usage infrastructure #37256 (issue: #37221)
  • [CCR] FollowingEngine should fail with 403 if operation has no seqno assigned #37213
  • [CCR] Added auto_follow_exception.timestamp field to auto follow stats #36947
  • [CCR] Add time since last auto follow fetch to auto follow stats #36542 (issues: #33007, #35895)

CRUD

  • Add Seq# based optimistic concurrency control to UpdateRequest #37872 (issues: #10708, #36148)
  • Introduce ssl settings to reindex from remote #37527 (issues: #29755, #37287)
  • Use Sequence number powered OCC for processing updates #37308 (issues: #10708, #36148)
  • Document Seq No powered optimistic concurrency control #37284 (issues: #10708, #36148)
  • Enable IPv6 URIs in reindex from remote #36874
  • Rename seq# powered optimistic concurrency control parameters to ifSeqNo/ifPrimaryTerm #36757 (issues: #10708, #36148)
  • Expose Sequence Number based Optimistic Concurrency Control in the rest layer #36721 (issues: #10708, #36148)
  • Add doc’s sequence number + primary term to GetResult and use it for updates #36680 (issues: #10708, #36148)
  • Add seq no powered optimistic locking support to the index and delete transport actions #36619 (issues: #10708, #36148)
  • Set acking timeout to 0 on dynamic mapping update #31140 (issues: #30672, #30844)

Distributed

  • Recover retention leases during peer recovery #38435 (issue: #37165)
  • Lift retention lease expiration to index shard #38380 (issues: #37165, #37963, #38070)
  • Introduce retention lease background sync #38262 (issue: #37165)
  • Allow shards of closed indices to be replicated as regular shards #38024 (issue: #33888)
  • Expose retention leases in shard stats #37991 (issue: #37165)
  • Introduce retention leases versioning #37951 (issue: #37165)
  • Soft-deletes policy should always fetch latest leases #37940 (issues: #37165, #37375)
  • Sync retention leases on expiration #37902 (issue: #37165)
  • Ignore shard started requests when primary term does not match #37899 (issue: #33888)
  • Move update and delete by query to use seq# for optimistic concurrency control #37857 (issues: #10708, #36148, #37639)
  • Introduce retention lease serialization #37447 (issues: #37165, #37398)
  • Add run under primary permit method #37440 (issue: #37398)
  • Introduce retention lease syncing #37398 (issue: #37165)
  • Introduce retention lease persistence #37375 (issue: #37165)
  • Add validation for retention lease construction #37312 (issue: #37165)
  • Introduce retention lease expiration #37195 (issue: #37165)
  • Introduce shard history retention leases #37167 (issue: #37165)
  • [Close Index API] Add unique UUID to ClusterBlock #36775
  • [Close Index API] Mark shard copy as stale if needed during shard verification #36755
  • [Close Index API] Propagate tasks ids between Freeze, Close and Verify Shard actions #36630
  • Always initialize the global checkpoint #34381

Engine

  • Specialize pre-closing checks for engine implementations #38702
  • Ensure that max seq # is equal to the global checkpoint when creating ReadOnlyEngines #37426
  • Enable Bulk-Merge if all source remains #37269
  • Rename setting to enable mmap #37070 (issue: #36668)
  • Add hybridfs store type #36668
  • Introduce time-based retention policy for soft-deletes #34943 (issue: #34908)
  • handle AsyncAfterWriteAction failure on primary in the same way as failures on replicas #31969 (issues: #31716, #31755)

Features/CAT APIs

  • Expose search.throttled on _cat/indices #37073 (issue: #34352)

Features/Features

  • Run Node deprecation checks locally (#38065) #38250 (issue: #38065)

Features/ILM

  • Ensure ILM policies run safely on leader indices #38140 (issue: #34648)
  • Skip Shrink when numberOfShards not changed #37953 (issue: #33275)
  • Inject Unfollow before Rollover and Shrink #37625 (issue: #34648)
  • Add set_priority action to ILM #37397 (issue: #36905)
  • [ILM] Add Freeze Action #36910 (issue: #34630)

Features/Indices APIs

  • New mapping signature and mapping string source fixed. #37401

Features/Ingest

  • ingest: compile mustache template only if field includes {{‘ #37207 (issue: #37120)
  • Move ingest-geoip default databases out of config #36949 (issue: #36898)
  • Make the ingest-geoip databases even lazier to load #36679
  • Updates the grok patterns to be consistent with the logstash #27181

Features/Java High Level REST Client

  • HLRC: Fix strict setting exception handling #37247 (issue: #37090)
  • HLRC: Use nonblocking entity for requests #32249

Features/Monitoring

  • Adding mapping for hostname field #37288

Features/Stats

  • Stats: Add JVM dns cache expiration config to JvmInfo #36372

Features/Watcher

  • Move watcher to use seq# and primary term for concurrency control #37977 (issues: #10708, #37872)
  • Use ILM for Watcher history deletion #37443 (issue: #32041)
  • Watcher: Add whitelist to HttpClient #36817 (issue: #29937)

Infra/Core

  • fix a few versionAdded values in ElasticsearchExceptions #37877
  • Add simple method to write collection of writeables #37448 (issue: #37398)
  • Date/Time parsing: Use java time API instead of exception handling #37222
  • [API] spelling: interruptible #37049 (issue: #37035)

Infra/Logging

Infra/Packaging

  • Add OS/architecture classifier to distributions #37881
  • Change file descriptor limit to 65535 #37537 (issue: #35839)
  • Exit batch files explictly using ERRORLEVEL #29583 (issue: #29582)

Infra/Scripting

  • Add getZone to JodaCompatibleZonedDateTime #37084
  • [Painless] Add boxed type to boxed type casts for method/return #36571

Infra/Settings

  • Separate out validation of groups of settings #34184

License

Machine Learning

  • Move ML Optimistic Concurrency Control to Seq No #38278 (issues: #10708, #36148)
  • [ML] Add explanation so far to file structure finder exceptions #38191 (issue: #29821)
  • ML: Add reason field in JobTaskState #38029 (issue: #34431)
  • [ML] Add _meta information to all ML indices #37964
  • ML: Add upgrade mode docs, hlrc, and fix bug #37942
  • [ML] Tighten up use of aliases rather than concrete indices #37874
  • ML: Add support for single bucket aggs in Datafeeds #37544 (issue: #36838)
  • [ML] Create the ML annotations index #36731 (issues: #26034, #33376)
  • [ML] Merge the Jindex master feature branch #36702 (issue: #32905)
  • [FEATURE][ML] Add cluster setting to enable/disable config migration #36700 (issue: #32905)

Mapping

  • Log document id when MapperParsingException occurs #37800 (issue: #37658)
  • [API] spelling: unknown #37056 (issue: #37035)
  • Make SourceToParse immutable #36971
  • Use index-prefix fields for terms of length min_chars - 1 #36703

Network

  • Enable TLSv1.3 by default for JDKs with support #38103 (issue: #32276)

Recovery

  • SyncedFlushService.getShardRoutingTable() should use metadata to check for index existence #37691 (issue: #33888)
  • Make prepare engine step of recovery source non-blocking #37573 (issue: #37174)
  • Make recovery source send operations non-blocking #37503 (issue: #37458)
  • Prepare to make send translog of recovery non-blocking #37458 (issue: #37291)
  • Make finalize step of recovery source non-blocking #37388 (issue: #37291)
  • Make recovery source partially non-blocking #37291 (issue: #36195)
  • Do not mutate RecoveryResponse #37204 (issue: #37174)
  • Don’t block on peer recovery on the target side #37076 (issue: #36195)
  • Reduce recovery time with compress or secure transport #36981 (issue: #33844)
  • Translog corruption marker #33415 (issue: #31389)

Rollup

  • Replace the TreeMap in the composite aggregation #36675

SQL

  • SQL: Allow look-ahead resolution of aliases for WHERE clause #38450 (issue: #29983)
  • SQL: Implement CURRENT_DATE #38175 (issue: #38160)
  • SQL: Generate relevant error message when grouping functions are not used in GROUP BY #38017 (issue: #37952)
  • SQL: Skip the nested and object field types in case of an ODBC request #37948 (issue: #37801)
  • SQL: Add protocol tests and remove jdbc_type from drivers response #37516 (issues: #36635, #36882)
  • SQL: Remove slightly used meta commands #37506 (issue: #37409)
  • SQL: Describe aliases as views #37496 (issue: #37422)
  • SQL: Make FULL non-reserved keyword in the grammar #37377 (issue: #37376)
  • SQL: Use declared source for error messages #37161
  • SQL: Improve error message when unable to translate to ES query DSL #37129 (issue: #37040)
  • [API] spelling: subtract #37055 (issue: #37035)
  • [API] spelling: similar #37054 (issue: #37035)
  • [API] spelling: input #37048 (issue: #37035)
  • SQL: Enhance message for PERCENTILE[_RANK] with field as 2nd arg #36933 (issue: #36903)
  • SQL: Preserve original source for each expression #36912 (issue: #36894)
  • SQL: Extend the ODBC metric by differentiating between 32 and 64bit platforms #36753 (issue: #36740)
  • SQL: Fix wrong appliance of StackOverflow limit for IN #36724 (issue: #36592)

Search

  • Tie break on cluster alias when merging shard search failures #38715 (issue: #38672)
  • Add finalReduce flag to SearchRequest #38104 (issues: #37000, #37838)
  • Streamline skip_unavailable handling #37672 (issue: #32125)
  • Expose sequence number and primary terms in search responses #37639
  • Add support for merging multiple search responses into one #37566 (issue: #32125)
  • Allow field types to optimize phrase prefix queries #37436 (issue: #31921)
  • Add support for providing absolute start time to SearchRequest #37142 (issue: #32125)
  • Ensure that local cluster alias is never treated as remote #37121 (issues: #32125, #36997)
  • [API] spelling: cacheable #37047 (issue: #37035)
  • Add ability to suggest shard_size on coord node rewrite #37017 (issues: #32125, #36997, #37000)
  • Skip final reduction if SearchRequest holds a cluster alias #37000 (issues: #32125, #36997)
  • Add support for local cluster alias to SearchRequest #36997 (issue: #32125)
  • Use SearchRequest copy constructor in ExpandSearchPhase #36772 (issue: #36641)
  • Add raw sort values to SearchSortValues transport serialization #36617 (issue: #32125)

Security

  • Move CAS operations in TokenService to sequence numbers #38311 (issues: #10708, #37872)
  • Cleanup construction of interceptors #38294
  • Add passphrase support to elasticsearch-keystore #37472 (issue: #32691)

Snapshot/Restore

  • RestoreService should update primary terms when restoring shards of existing indices #38177 (issue: #33888)
  • Allow open indices to be restored #37733
  • Create specific exception for when snapshots are in progress #37550 (issue: #37541)
  • SNAPSHOT: Make Atomic Blob Writes Mandatory #37168 (issues: #37011, #37066)
  • SNAPSHOT: Speed up HDFS Repository Writes #37069
  • Implement Atomic Blob Writes for HDFS Repository #37066 (issue: #37011)
  • [API] spelling: repositories #37053 (issue: #37035)
  • SNAPSHOT: Use CancellableThreads to Abort #35901 (issue: #21759)
  • WIP: S3 client encryption #30513 (issues: #11128, #16843)

Suggesters

  • Remove unused empty constructors from suggestions classes #37295
  • [API] spelling: likelihood #37052 (issue: #37035)

ZenDiscovery

  • Add elasticsearch-node detach-cluster tool #37979
  • Deprecate minimum_master_nodes #37868
  • Step down as master when configured out of voting configuration #37802 (issue: #37712)
  • Enforce cluster UUIDs #37775
  • Bubble exceptions up in ClusterApplierService #37729
  • Use m_m_nodes from Zen1 master for Zen2 bootstrap #37701
  • Add tool elasticsearch-node unsafe-bootstrap #37696
  • Report terms and version if cluster does not form #37473
  • Bootstrap a Zen2 cluster once quorum is discovered #37463
  • Zen2: Add join validation #37203
  • Publish cluster states in chunks #36973

Bug fixes

Aggregations

  • Don’t load global ordinals with the map execution_hint #37833 (issue: #37705)
  • Issue #37303 - Invalid variance fix #37384 (issue: #37303)

Allocation

  • Fix _host based require filters #38173
  • ALLOC: Fail Stale Primary Alloc. Req. without Data #37226 (issue: #37098)

Audit

Authentication

  • Enhance parsing of StatusCode in SAML Responses #38628
  • Limit token expiry to 1 hour maximum #38244
  • Fix expired token message in Exception header #37196
  • Fix NPE in CachingUsernamePasswordRealm #36953 (issue: #36951)

CCR

  • Prevent CCR recovery from missing documents #38237
  • Fix file reading in ccr restore service #38117
  • Correct argument names in update mapping/settings from leader #38063
  • Ensure changes requests return the latest mapping version #37633
  • Do not set fatal exception when shard follow task is stopped. #37603
  • Add fatal_exception field for ccr stats in monitoring mapping #37563
  • Do not add index event listener if CCR disabled #37432
  • When removing an AutoFollower also mark it as removed. #37402 (issue: #36761)
  • [CCR] Make shard follow tasks more resilient for restarts #37239 (issue: #37231)
  • [CCR] Resume follow Api should not require a request body #37217 (issue: #37022)
  • [CCR] Report error if auto follower tries auto follow a leader index with soft deletes disabled #36886 (issue: #33007)
  • Remote cluster license checker and no license info. #36837 (issue: #36815)
  • Make CCR resilient against missing remote cluster connections #36682 (issues: #36255, #36667)
  • [CCR] AutoFollowCoordinator and follower index already created #36540 (issue: #33007)

CRUD

  • Fix Reindex from remote query logic #36908
  • Synchronize WriteReplicaResult callbacks #36770

Distributed

  • TransportVerifyShardBeforeCloseAction should force a flush #38401 (issues: #33888, #37961)
  • Fix limit on retaining sequence number #37992 (issue: #37165)
  • Close Index API should force a flush if a sync is needed #37961 (issues: #33888, #37426)
  • Force Refresh Listeners when Acquiring all Operation Permits #36835
  • Replaced the word shards with replicas in an error message. (#36234) #36275 (issue: #36234)

Engine

  • Subclass NIOFSDirectory instead of using FileSwitchDirectory #37140 (issues: #36668, #37111)

Features/ILM

  • Preserve ILM operation mode when creating new lifecycles #38134 (issues: #38229, #38230)
  • Retry ILM steps that fail due to SnapshotInProgressException #37624 (issues: #37541, #37552)
  • Remove indexing_complete when removing policy #36620

Features/Indices APIs

Features/Ingest

  • Support unknown fields in ingest pipeline map configuration #38352 (issue: #36938)
  • Ingest node - user_agent, move device parsing to an object #38115 (issues: #37329, #38094)
  • ingest: fix on_failure with Drop processor #36686 (issue: #36151)
  • ingest: support default pipelines + bulk upserts #36618 (issue: #36219)

Features/Java High Level REST Client

  • Update IndexTemplateMetadata to allow unknown fields #38448 (issue: #36938)
  • if_seq_no and if_primary_term parameters aren’t wired correctly in REST Client’s CRUD API #38411
  • Update Rollup Caps to allow unknown fields #38339 (issue: #36938)
  • Fix ILM explain response to allow unknown fields #38054 (issue: #36938)
  • Fix ILM status to allow unknown fields #38043 (issue: #36938)
  • Fix ILM Lifecycle Policy to allow unknown fields #38041 (issue: #36938)
  • Update authenticate to allow unknown fields #37713 (issue: #36938)
  • Update verify repository to allow unknown fields #37619 (issue: #36938)
  • Update get users to allow unknown fields #37593 (issue: #36938)
  • Update Execute Watch to allow unknown fields #37498 (issue: #36938)
  • Update Put Watch to allow unknown fields #37494 (issue: #36938)
  • Update Delete Watch to allow unknown fields #37435 (issue: #36938)
  • Fix rest reindex test for IPv4 addresses #37310
  • Fix weighted_avg parser not found for RestHighLevelClient #37027 (issue: #36861)

Features/Java Low Level REST Client

  • Fix potential IllegalCapacityException in LLRC when selecting nodes #37821

Features/Monitoring

  • Allow built-in monitoring_user role to call GET _xpack API #38060 (issue: #37970)

Features/Watcher

  • Support merge nested Map in list for JIRA configurations #37634 (issue: #30068)
  • Watcher accounts constructed lazily #36656
  • Ensures watch definitions are valid json #30692 (issue: #29746)

Geo

  • Fix GeoHash PrefixTree BWC #38584 (issue: #38494)
  • Geo: Do not normalize the longitude with value -180 for Lucene shapes #37299 (issue: #37297)

Infra/Core

  • Bubble-up exceptions from scheduler #38317 (issue: #38014)
  • Core: Revert back to joda’s multi date formatters #36814 (issues: #36447, #36602)
  • Propagate Errors in executors to uncaught exception handler #36137 (issue: #28667)

Infra/Packaging

  • Remove NOREPLACE for /etc/elasticsearch in rpm and deb #37839
  • Packaging: Update marker used to allow ELASTIC_PASSWORD #37243 (issue: #37240)
  • Packaging: Remove permission editing in postinst #37242 (issue: #37143)

Infra/REST API

Infra/Scripting

  • Fix Painless void return bug #38046

Infra/Settings

  • Fix setting by time unit #37192
  • Fix handling of fractional byte size value settings #37172
  • Fix handling of fractional time value settings #37171

Machine Learning

  • [ML] Report index unavailable instead of waiting for lazy node #38423
  • ML: Fix error race condition on stop _all datafeeds and close _all jobs #38113 (issue: #37959)
  • [ML] Update ML results mappings on process start #37706 (issue: #37607)
  • [ML] Prevent submit after autodetect worker is stopped #37700 (issue: #37108)
  • [ML] Fix ML datafeed CCS with wildcarded cluster name #37470 (issue: #36228)
  • [ML] Update error message for process update #37363
  • [ML] Wait for autodetect to be ready in the datafeed #37349 (issues: #36810, #37227)
  • [ML] Stop datafeeds running when their jobs are stale #37227 (issue: #36810)
  • [ML] Order GET job stats response by job id #36841 (issue: #36683)
  • [ML] Make GetJobStats work with arbitrary wildcards and groups #36683 (issue: #34745)

Mapping

  • Treat put-mapping calls with _doc as a top-level key as typed calls. #38032
  • Correct deprec log in RestGetFieldMappingAction #37843 (issue: #37667)
  • Restore a noop _all metadata field for 6x indices #37808 (issue: #37429)
  • Make sure PutMappingRequest accepts content types other than JSON. #37720
  • Make sure to use the resolved type in DocumentMapperService#extractMappings. #37451 (issue: #36811)
  • MAPPING: Improve Precision for scaled_float #37169 (issue: #32570)
  • Make sure to accept empty unnested mappings in create index requests. #37089
  • Stop automatically nesting mappings in index creation requests. #36924
  • Rewrite SourceToParse with resolved docType #36921 (issues: #35790, #36769)

Network

Ranking

  • QueryRescorer should keep the window size when rewriting #36836

Recovery

  • RecoveryMonitor#lastSeenAccessTime should be volatile #36781

Rollup

SQL

  • SQL: Prevent grouping over grouping functions #38649 (issue: #38308)
  • SQL: Relax StackOverflow circuit breaker for constants #38572 (issue: #38571)
  • SQL: Fix issue with IN not resolving to underlying keyword field #38440 (issue: #38424)
  • SQL: change the Intervals milliseconds precision to 3 digits #38297 (issue: #37423)
  • SQL: Fix esType for DATETIME/DATE and INTERVALS #38179 (issue: #38051)
  • SQL: Added SSL configuration options tests #37875 (issue: #37711)
  • SQL: Fix casting from date to numeric type to use millis #37869 (issue: #37655)
  • SQL: Fix BasicFormatter NPE #37804
  • SQL: Return Intervals in SQL format for CLI #37602 (issues: #29970, #36186, #36432)
  • SQL: fix object extraction from sources #37502 (issue: #37364)
  • SQL: Fix issue with field names containing “.” #37364 (issue: #37128)
  • SQL: Fix bug regarding alias fields with dots #37279 (issue: #37224)
  • SQL: Proper handling of COUNT(field_name) and COUNT(DISTINCT field_name) #37254 (issue: #30285)
  • SQL: fix COUNT DISTINCT filtering #37176 (issue: #37086)
  • SQL: Fix issue with wrong NULL optimization #37124 (issue: #35872)
  • SQL: Fix issue with complex expression as args of PERCENTILE/_RANK #37102 (issue: #37099)
  • SQL: Handle the bwc Joda ZonedDateTime scripting class in Painless #37024 (issue: #37023)
  • SQL: Fix bug regarding histograms usage in scripting #36866
  • SQL: Fix issue with always false filter involving functions #36830 (issue: #35980)
  • SQL: protocol returns ISO 8601 String formatted dates instead of Long for JDBC/ODBC requests #36800 (issue: #36756)
  • SQL: Enhance Verifier to prevent aggregate or grouping functions from #36799 (issue: #36798)
  • SQL: Fix translation of LIKE/RLIKE keywords #36672 (issues: #36039, #36584)
  • SQL: Scripting support for casting functions CAST and CONVERT #36640 (issue: #36061)
  • SQL: Concat should be always not nullable #36601 (issue: #36169)
  • SQL: Fix issue with complex HAVING and GROUP BY ordinal #36594 (issue: #36059)

Search

  • Look up connection using the right cluster alias when releasing contexts #38570
  • Fix fetch source option in expand search phase #37908 (issue: #23829)
  • Change rational to saturation in script_score #37766 (issue: #37714)
  • Throw if two inner_hits have the same name #37645 (issue: #37584)
  • Ensure either success or failure path for SearchOperationListener is called #37467 (issue: #37185)
  • query_string should use indexed prefixes #36895
  • Avoid duplicate types deprecation messages in search-related APIs. #36802

Security

  • Fix exit code for Security CLI tools #37956 (issue: #37841)
  • Fix potential NPE in UsersTool #37660

Snapshot/Restore

  • Fix Concurrent Snapshot Ending And Stabilize Snapshot Finalization #38368 (issue: #38226)
  • Fix Two Races that Lead to Stuck Snapshots #37686 (issues: #32265, #32348)
  • Fix Race in Concurrent Snapshot Delete and Create #37612 (issue: #37581)
  • Streamline S3 Repository- and Client-Settings #37393

Suggesters

  • Fix duplicate removal when merging completion suggestions #36996 (issue: #35836)

Task Management

  • Un-assign persistent tasks as nodes exit the cluster #37656

ZenDiscovery

  • Fix size of rolling-upgrade bootstrap config #38031
  • Always return metadata version if metadata is requested #37674
  • [Zen2] Elect freshest master in upgrade #37122 (issue: #40)
  • Fix cluster state persistence for single-node discovery #36825

Regressions

Infra/Core

  • Restore date aggregation performance in UTC case #38221 (issue: #37826)
  • Speed up converting of temporal accessor to zoned date time #37915 (issue: #37826)

Mapping

  • Performance fix. Reduce deprecation calls for the same bulk request #37415 (issue: #37411)

Upgrades

Engine

  • Upgrade to lucene-8.0.0-snapshot-83f9835. #37668

Machine Learning

  • [ML] No need to add state doc mapping on job open in 7.x #37759