1.5. 2.2.x Branch

1.5.1. Upgrade Notes

  • The minimum supported version of Erlang is now 17, not R16B03. Support for Erlang 21 is still ongoing and will be provided in a future release.

  • The CouchDB replication client can now use the /_session endpoint when authenticating against remote CouchDB instances, improving performance since re-authorization does not have to be performed with every request. Because of this performance improvement, it is recommended to increase the PBKDF2 work factor beyond the default 10 to a modern default such as 10000. This is done via the local ini file setting [couch_httpd_auth] iterations = 10000.

    Do not do this if an older version of CouchDB is replicating TO this instance or cluster regularly, since CouchDB < 2.2.0 must perform authentication on every request and replication performance will suffer.

    A future version will make this increased number of iterations a default.

  • #820, #1032: Multiple queries can now be made at the POST /{db}/_all_docs/queries, POST /{db}/_design_docs/queries and POST /{db}/_local_docs/queries endpoints. Also, a new endpoint POST /{db}/_design/{ddoc}/_view/{view}/queries has been introduced to replace the ?queries parameter formerly provided for making multiple queries to a view. The old ?queries parameter is now deprecated and will be removed in a future release of CouchDB.

  • The maximum http request limit, which had been lowered in 2.1.0, has been re-raised to a 4GB limit for now. (#1446). Ongoing discussion about the path forward for future releases is available in #1200 and #1253.

  • #1118: The least recently used (LRU) cache of databases is now only updated on database write, not read. This has lead to significant performance enhancements on very busy clusters. To restore the previous behaviour, your local ini file can contain the block [couchdb] update_lru_on_read = true.

  • #1153: The CouchDB replicator can now make use of the /_session endpoint rather than relying entirely on HTTP basic authentication headers. This can greatly improve replication performance. We encourage you to upgrade any nodes or clusters that regularly act as replication clients to use this new feature, which is enabled by default (#1462).

  • #1283: The [couchdb] enable_database_recovery feature, which only soft-deletes databases in response to a DELETE /{db} call, is now documented in default.ini.

  • #1330: CouchDB externals and OS daemons are now officially deprecated and no longer documented. Support for these features will be completely removed in a future release of CouchDB (probably 3.0.0).

  • #1436: CouchDB proxy authentication now uses a proper chttpd_auth module, simplifying configuration in local ini files. While this is not a backward- compatible breaking change, it is best to update your local ini files to reference the new {chttpd_auth, proxy_authentication_handler} handler rather than the couch_httpd_auth version, as couch_httpd is in the process of being deprecated completely.

  • #1476, #1477: The obsolete update_notification feature, which was replaced by /{db}/_changes feeds c. CouchDB 1.2, has been completely removed. This feature never worked in 2.0 for databases, only for shards, making it effectively useless.

1.5.2. Version 2.2.0

1.5.2.1. Features

  • Much improved documentation. Highlights include:

    • A complete rewrite of the sharding documentation.
    • Developer installation notes (INSTALL.*.rst)
    • Much of the content of the original CouchDB Wiki has been imported into the official docs. (The old CouchDB Wiki is in the process of being deprecated.)

  • Much improved Fauxton functionality. Highlights include:

    • Search support in the code editor
    • Support for relative Fauxton URLs (i.e., not always at /_utils)
    • Replication setup enhancements for various authentication mechanisms
    • Fixes for IE10, IE11, and Edge (we hope…)
    • Resolving conflicts of design documents is now allowed

  • #496, COUCHDB-3287: New pluggable storage engine framework has landed in CouchDB. This internal refactor makes it possible for CouchDB to use different backends for storing the base database file itself. The refactor included a full migration of the existing “legacy” storage engine into the new framework.

  • #603: When creating a new database on a cluster without quorum, CouchDB will now return a 202 Accepted code if possible, indicating that at least one node has written the database record to disk, and that other nodes will be updated as they return to an online state. This replaces the former 500 internal error.

  • #1136, #1139: When deleting a database in a cluster without quorum, CouchDB will no longer throw a 500 error status, but a 202 as long as at least one node records the deletion, or a 200 when all nodes respond. This fix parallels the one made for #603.

  • #745: CouchDB no longer fails to complete replicating databases with large attachments. The fix for this issue included several related changes:

    • The maximum http request limit, which had been lowered in 2.1.0, has been re-raised to a 4GB limit for now. (#1446). Ongoing discussion about the path forward for future releases is available in #1200 and #1253.
    • An update to the replicator http client that improves active socket accounting, without which CouchDB can cease to be responsive over the main http interface (#1117)
    • The replicator’s http client no longer performs unconditional retries on failure (#1177)
    • A path by which CouchDB could lose track of their RPC workers during multipart attachment processing was removed. (#1178)
    • When CouchDB transmits a 413 Payload Too Large response on attachment upload, it now correctly flushes the receive socket before closing the connection to avoid a TCP reset, and to give the client a better chance of parsing the 413 response. In tandem, the replicator http client correctly closes its own socket after processing any 413 response. (#1234)
    • A fabric process to receive unchunked attachments can no longer orphan processes that leave unprocessed binaries in memory until all available memory is exhausted. (#1264).
    • When using CouchDB’s native SSL responder (port 6984 by default), sessions are now timed out by default after 300s. This is to work around RAM explosion in the BEAM VM when using the Erlang-native SSL libraries. (#1321

  • #822: A new end point /_dbs_info has been added to return information about a list of specified databases. This endpoint can take the place of multiple queries to /{db}.

  • #875, #1030: couch_peruser installations can now specify a default q value for each peruser-created database that is different from the cluster’s q value. Set this in your local ini file, under [couch_peruser] q.

  • #876, #1068: The couch_peruser database prefix is now configurable through your local ini file, under [couch_peruser] database_prefix.

  • #887: Replicator documents can now include parameters for target database creation, such as "create_target_params": {"q": "1"}. This can assist in database resharding or placement.

  • #977: When using COPY to copy a document, CouchDB no longer fails if the new ID includes Unicode characters.

  • #1095: Recognize the environment variables ARGS_FILE, SYSCONFIG_FILE, COUCHDB_ARGS_FILE and COUCHDB_SYSCONFIG_FILE to overrride where CouchDB looks for the vm.args and sys.config files at startup.

  • #1101, #1425: Mango can now be used to find conflicted documents in a database by adding conflicts: true to a mango selector.

  • #1126: When queried back after saving, replication documents no longer contain sensitive credential information (such as basic authenticataion headers).

  • #1203:

    • The compaction daemon now has a snooze period, during which it waits to start the next compaction after finishing the previous one. This value is useful in setups with many databases (e.g. with couch_peruser) or many design docs, which can cause a CPU spike every check_interval seconds. The setting can be adjusted in your local ini file via [compaction_daemon] snooze_period. The current default is a 3 second pause.
    • The check_interval has been raised from 300 seconds to 3600 seconds.
    • A notice-level log about closing view indexes has been demoted to the debug level. In a sceario with many design docs, this would createsignficant load on the logging subsystem every [compaction_daemon] check_interval for no discernible benefit.

  • #1309, #1435: CouchDB now reports the git sha at the time of build in the top-level GET / version string, in a new git_sha key. This can be used to help ensure an unmodified version of CouchDB has been built and is running on any given machine.

  • COUCHDB-2971, #1346: CouchDB now includes a new builtin reduce function _approx_count_distinct, that uses a HyperLogLog algorithm to estimate the number of distinct keys in the view index. The precision is currently fixed to 2^11 observables, and therefore uses approximately 1.5KB of memory.

  • #1377: CouchDB finalization of view reduces now occurs at the coordinator node. This simplified the built-in _stats function.

  • #1392: When running CouchDB under Erlang 19.0 or newer, messages can now be stored off the process heap. This is extremely useful for Erlang processes that can have huge number of messages in their mailbox, and is now enabled for couch_server, couch_log_server, ddoc_cache, mem3_shards, and rexi_server whenever possible.

  • #1424: The CouchDB native SSL/TLS server httpsd now accepts socket-level configuration options through the [httpsd] server_options ini file setting.

  • #1440: CouchDB can now be configured to prevent non-admins from accessing the GET /_all_dbs method by specifying [chttpd] admin_only_all_dbs = true in your local ini file(s). The true setting will become default in future versions.

  • #1171, #1445: CouchDB can now be configured to use the internal Erlang MD5 hash function when not available in the external environment (e.g. FIPS enabled CentOS) at compile time with the configure flag --enable-md5. Because this implementation is slower, it is not recommended in the general case.

1.5.2.2. Performance

  • #958: The revision stemming algorithm was optimized down from O(N^2) to O(N) via a depth-first search approach, and then further improved by calling the stemming operation only when necessary. This new algorithm can be disabled by setting the option [couchdb] stem_interactive_updates = false if necessary.

  • #1246: CouchDB now checks for request authorization only once per each database request, improving the performance of any request that requires authorization.

1.5.2.3. Bugfixes

  • #832, #1064: Tracking of Couch logging stats has been added back into the per-node /_node/<node-name>/_stats endpoint.

  • #953, #973: Return 404 Not Found on GET /_scheduler, not 405 Method Not Allowed.

  • #955: The /{db}/_bulk_docs endpoint now correctly responds with a 400 Bad Request error if the new_edits parameter is not a boolean.

  • #969: CouchDB now returns offset and update_seq values when keys are provided to the GET or POST /{db}/_all_docs?update_seq=true endpoints. This was affecting PouchDB compatibility.

  • #984, #1434: CouchDB views now retain their update_seq after compaction, preventing potentially expensive client-side view rewinds after compaction.

  • #1012: Address a theoretical race condition the replication scheduler could encounter when trying to determine if the cluster is “stable” enough to resume handling replication-introduced document updates.

  • #1051: Return a user-friendly error message when attempting to create a CouchDB user with an invalid password field (non-string).

  • #1059: DB-specific compaction configurations were not working correctly. The syntax now also supports shard-level custom compaction configuration if desired (which it probably isn’t.)

  • #1097: Compaction daemon will not crash out when trying to check specific file system mounts that are not “real” file systems (like /run on Linux).

  • #1198: Fauxton is no longer available on the node-local port (5986, by default). The node-local port is only to be used for specific administrative tasks; removing the Fauxton interface prevents mistaking the node-local port as the correct CouchDB port (5984, by default).

  • #1165: validate_doc_update view functions can once again be implemented directly in Erlang (after enabling the optional Erlang view server).

  • #1223: The couch_config application now correctly handles non-persistent integer and boolean-valued configuration changes.

  • #1242: couch_os_daemons may now reside in directories with spaces.

  • #1258: CouchDB will now successfully login users, even if password encryption is very slow.

  • #1276: The replication scheduler status for a repeatedly erroring job now correctly reflects the crashing state in more scenarios.

  • #1375: If CouchDB fails authorization but passes authentication, it no longer drops the user_ctx out of the request.

  • #1390: The active size of views (as returned in a database info response) no longer is incorrectly calculated in such a way that it could occasionally be larger than the actual on-disk file size.

  • #1401: CouchDB Erlang views no longer crash in the couch_native process with an unexpected function_clause error.

  • #1419: When deleting a file, CouchDB now properly ignores the configuration flag enable_database_recovery when set when compacting databases, rather than always retaining the old, renamed, uncompacted database file.

  • #1439: The CouchDB setup wizard now correctly validates bind_addresses. It also no longer logs credentials by moving logging of internal wizard setup steps to the debug level from the notice level.

1.5.2.4. Mango

  • #816, #962, #1038: If a user specifies a value for use_index that is not valid for the selector (does not meet coverage requirements or proper sort fields), attempt to fall back to a valid index or full DB scan rather than returning a 400. If we fall back, populate a warning field in the response. Mango also tries to use indexes where $or may select a field only when certain values are present.

  • #849: When {"seq_indexed": true} is specified, a badmatch error was returned. This is now fixed.

  • #927, #1310: Error messages when attempting to sort incorrectly are now actually useful.

  • #951: When using GET /{db}/_index, only use a partial filter selector for an index if it is set to something other than the default.

  • #961: Do not prefix _design/ to a Mango index name whose user-specified name already starts with _design/.

  • #988, #989: When specifying a use_index value with an invalid index, correctly return a 400 Bad Request showing that the requested index is invalid for the request specified.

  • #998: The fix for CVE 2017-12635 presented a breaking change to Mango’s /{db}/_find, which would evaluate all instances of all JSON fields in a selector. Mango is now tested to ensure it only considers the last instance of a field, silently ignoring those that appear before it.

  • #1014: Correctly deduce list of indexed fields in a selector when nested $and operators are specified.

  • #1023: Fix an unexpected 500 error if startkey and endkey in a Mango selector were reversed.

  • #1067: Prevent an invalid_cast crash when the couch_proc_manager soft limit for processes is reached and mango idle processes are stopped.

  • #1336: The built-in fields _id and rev will always be covered by any index, and Mango now correctly ignores their presence in any index that explicitly includes them for selector matching purposes.

  • #1376: Mango now appropriately selects some indexes as usable for queries, even if not all columns for an index are added to the query’s sort field list.

  • Multiple fixes related to using Mango as a front-end for full text indexing (a feature not shipped with couch, but for which support is in place as a compile-time addon).

1.5.2.5. Other

The 2.2.0 release also includes the following minor improvements:

  • Developers can, at build time, enable curl libraries & disable Fauxton and documentation builds by specifying the new --dev option to the configure script.

  • The mochiweb dependency was bumped to version 2.17.0, in part to address the difficult #745 issue.

  • Improved compatibility with newer versions of Erlang (20.x)

  • Improved release process for CouchDB maintainers and PMC members.

  • Multiple test suite improvements, focused on increased coverage, speed, and reliability.

  • Improvements to the Travis CI and Jenkins CI setups, focused on improved long-term project maintenance and automatability.

  • Related improvements to the CouchDB deb/rpm packaging and Docker repositories to make deployment even easier.

  • #1007: Move etc/default.ini entries back into [replicator] section (incorrectly moved to [couch_peruser] section)

  • #1245: Increased debug-level logging for shard open errors is now available.

  • #1296: CouchDB by default now always invokes the SMP-enabled BEAM VM, even on single-processor machines. A future release of Erlang will remove the non-SMP BEAM VM entirely.

  • A pony! OK, no, not really. If you got this far…thank you for reading.