Elasticsearch version 8.0.0-beta1

Elasticsearch version 8.0.0-beta1

Also see Breaking changes in 8.0.

Known issues

  • If you’re using machine learning, it’s not safe to upgrade to 8.0.0-beta1 if the cluster you’re upgrading was first used prior to 7.7.0. If you attempt such an upgrade the filtered aliases against the machine learning results indices will lose their filters, causing machine learning anomaly detection results for different jobs to be mixed. Wait for 8.0.0-rc1 before attempting to test upgrading a cluster where machine learning was used prior to version 7.7.0.
  • System indices are included in wildcard queries when using the * pattern. Do not rely on this behavior, because system indices will be hidden in the 8.0 release. If you need to access system indices, specify the index name, use a more specific wildcard, or use the expand_wildcards parameter in your query.
  • By default, system indices are included in snapshots of all indices, and are restored when no indices are specified. This behavior results from including system indices in the implicit default wildcard (*) for snapshot and restore operations, which was intended for backwards compatibility in 7.x versions. In 8.0, all system indices should be accessed through the feature_states parameter of the snapshot or restore request.

Breaking changes

ILM+SLM

  • Always enforce default tier preference #79751 (issue: #76147)
  • Validate that snapshot repository exists for ILM policies at creation/update time #78468 (issues: #72957, #77657)
  • Default ENFORCE_DEFAULT_TIER_PREFERENCE to true #79275 (issues: #76147, #79210)

Indices APIs

Infra/Core

  • Remove Joda dependency #79007
  • Remove Joda support from date formatters #78990

Ingest

  • Remove default maxmind GeoIP databases from distribution #78362 (issue: #68920)

License

  • Enforce license expiration #79671

Machine Learning

  • Remove allow_no_datafeeds and allow_no_jobs parameters from APIs #80048 (issue: #60732)

Packaging

  • Require Java 17 for running Elasticsearch #79873

Deprecations

Authentication

  • Deprecate setup-passwords tool #76902

CRUD

  • Remove indices_segments verbose parameter #78451 (issue: #75955)

Monitoring

  • Add deprecation info API entries for deprecated monitoring settings #78799
  • Automatically install monitoring templates at plugin initialization #78350
  • Remove Monitoring ingest pipelines #77459 (issue: #50770)

New features

Security

Enhancements

Authentication

  • New CLI tool to reset password for built-in users #79709
  • Upgrade to UnboundID LDAP SDK v6.0.2 #79332
  • Auto-configure the elastic user password #78306

Cluster Coordination

Data streams

  • Data stream support read and write with custom routing and partition size #74394 (issue: #74390)

ILM+SLM

  • Inject migrate action regardless of allocate action #79090 (issue: #76147)

Infra/Core

  • Check whether stdout is a real console #79882
  • Share int, long, float, double, and byte pages #75053
  • Revert “Deprecate resolution loss on date field (#78921)” #79914 (issue: #78921)
  • Add two missing entries to the deprecation information API #80290 (issue: #80233)

Infra/Scripting

Ingest

  • Remove binary field after attachment processor execution #79172
  • Improving cache lookup to reduce recomputing / searches #77259

License

  • Enforce Transport TLS check on all licenses #79602 (issue: #75292)

Machine Learning

  • Add inference time configuration overrides #78441 (issue: #77799)
  • Optimize source extraction for categorize_text aggregation #79099
  • The Linux build platform for the machine learning C++ code is now CentOS 7 running gcc 10.3. #2028
  • Make ML indices hidden when the node becomes master #77416 (issue: #53674)

Mapping

  • Add support for configuring HNSW parameters #79193 (issue: #78473)
  • Extend dense_vector to support indexing vectors #78491 (issue: #78473)

Monitoring

  • Add previously removed Monitoring settings back for 8.0 #78784
  • Change Monitoring plugin cluster alerts to not install by default #79657

Packaging

Search

  • Node level can match action #78765
  • TSDB: Add time series information to field caps #78790 (issue: #74660)
  • Add new kNN search endpoint #79013 (issue: #78473)
  • Disallow kNN searches on nested vector fields #79403 (issue: #78473)
  • Ensure kNN search respects authorization #79693 (issue: #78473)
  • Load kNN vectors format with mmapfs #78724 (issue: #78473)
  • Support cosine similarity in kNN search #79500
  • Node level can match action #78765

Security

  • Add v7 restCompat for invalidating API key with the id field #78664 (issue: #66671)
  • Print enrollment token on startup #78293
  • Startup check for security implicit behavior change #76879
  • Update auto-generated credentials output #79755 (issue: #79312)
  • CLI tool to reconfigure nodes to enroll #79690 (issue: #7718)
  • Security auto-configuration for packaged installations #75144 (issue: #78306)
  • Update to OpenSAML 4 #77012 (issue: #71983)

Snapshot/Restore

  • Allow listing older repositories #78244
  • Optimize SLM Policy Queries #79341 (issue: #79321)
  • Upgrade repository-hdfs plugin to Hadoop 3 #76897

Transform

  • Prevent old beta transforms from starting #79712

TSDB

  • Automatically add timestamp mapper #79136
  • Create a coordinating node level reader for tsdb #79197
  • Fix TSDB shrink test in multi-version cluster #79940 (issue: #79936)
  • Do not allow shadowing metrics or dimensions #79757

Bug fixes

Infra/Core

  • Prevent stack overflow in rounding #80450

Infra/Settings

  • Stricter UpdateSettingsRequest parsing on the REST layer #79227 (issue: #29268)
  • Set Auto expand replica on deprecation log data stream #79226 (issue: #78991)

Machine Learning

  • Add timeout parameter for delete trained models API #79739 (issue: #77070)
  • Fix MlMetadata backwards compatibility bug with 7.13 through 7.16 #80041
  • Tone down ML unassigned job notifications #79578 (issue: #79270)
  • Use a new annotations index for future annotations #79006 (issue: #78439)

Search

  • Remove unsafe assertion in wildcard field #78966

Snapshot/Restore

  • Don’t fill stack traces in SnapshotShardFailure #80009 (issue: #79718)

Regressions

Search

  • Disable numeric sort optimization conditionally #78103

Upgrades

Search