16.3. 2.1.x Branch
16.3.1. Upgrade Notes
- When upgrading from 2.x to 2.1.1, if you have not customized yournode name in
vm.args
, be sure to retain your originalvm.args
file. The default node name has changed fromcouchdb@localhost
tocouchdb@127.0.0.1
, which can prevent CouchDB from accessing existingdatabases on the system. You may also change the name option back to theold value by setting-name couchdb@localhost
inetc/vm.args
byhand. The default has changed to meet new guidelines and to provideadditional functionality in the future.
If you receive errors in the logfile, such asinternal_server_error : No DB shards could be opened.
or in Fauxton,such as This database failed to load.
you need to make this change.
The deprecated (and broken) OAuth 1.0 implementation has been removed.
If user code reads or manipulates replicator document states,consider using the
[replicator] update_docs = true
compatibilityparameter. In that case the replicator will continue updating documentswith transient replication states. However, that will incur aperformance cost. Consider instead using the_scheduler/docs
HTTPendpoint.The
stale
parameter for views and_find
has been deprecated in favourof two new parameters:stable
andupdate
. The oldstale=ok
behaviour is equivalent tostable=true&update=false
, and the oldstale=update_after
behaviour is equivalent tostable=true&update=lazy
.The deprecatedstale
parameter will be removed in CouchDB 3.0.The new
httpd/max_http_request_size
configuration parameterwas added. This has the same behavior as the oldcouchdb/max_document_size
configuration parameter, whichhad been unfortunately misnamed, and has now been updated to behave as thename would suggest. Both are documented in the shippeddefault.ini
file.
Note that the default for this new parameter is 64MB instead of 4GB. If youget errors when trying to PUT or POST and see HTTP 413 return codes in couchdblogs, this could be the culprit. This can affect couchup in-place upgrades aswell.
#914: Certain critical config sections are blacklisted from beingmodified through the HTTP API. These sections can still be modified throughthe standard
local.ini
orlocal.d/*.ini
files.#916:
couchjs
now disableseval()
and theFunction()
constructor by default. To restore the original behaviour, add the—eval
flag to the definition of the javascript query server in yourlocal.ini
file.
16.3.2. Version 2.1.2
16.3.2.1. Security
16.3.3. Version 2.1.1
16.3.3.1. Security
16.3.3.2. General
- #617: CouchDB now supports compilation and running under Erlang/OTP20.x.
- #756: The
couchperuser
functionality is now _really fixed.Really. - #827: The cookie domain for AuthSession cookies, used in aproxy authentication configuration, can now be customized via the ini file.
- #858: It is now possible to modify shard maps for system databases.
- #732: Due to an Erlang bug (ERL-343), invalid paths can bereturned if volumes are mounted containing whitespace in their name. Thisproblem surfaced primarily on macOS (Time Machine volumes). CouchDB nowworks around this bug in unpatched versions of Erlang by skipping the freespace check performed by the compaction daemon. Erlang itself willcorrectly perform free space checks in version 21.0.
- #824: The current node’s local interface can now be accessed at
/_node/_local/{endpoint}
as well as at/_node/<nodename>@<hostname>/{endpoint}
. - The Dockerfile in the source repository has been retired. For a currentDockerfile, see the couchdb-docker repository.
- Fauxton now uses a version of React with a BSD license.
16.3.3.3. Performance
- #835: CouchDB now no longer decompresses documents just todetermine their uncompressed size. In tests, this has lead to improvementsbetween 10-40% in both CPU and wall-clock time for database compaction.
- The design document cache (
ddoc_cache
) has been rewritten to improveperformance.
16.3.3.4. Mango
- #808: Mango now supportspartial indexes. Partial indexes allowdocuments to be filtered at indexing time, potentially offeringsignificant performance improvements for query selectors that don’t mapcleanly to a range query on an index.
- #740: Mango queries can now be paginated. Each query responseincludes a bookmark. The bookmark can be provided on a subsequent query tocontinue from a specific key.
- #768: Mango
_find
accepts anexecution_stats
parameter. If present, a new object is included in the response whichcontains information about the query executed. The object contains thecount of total keys examined (0 for json indexes), total documentsexamined (wheninclude_docs=true
is used), and the total quorumdocuments examined (when fabric doc lookups are used). - #816 and #866: Mango now requires that all of the fieldsin a candidate index must exist in a query’s selector. Previously, this checkwas incorrect, and indexes that might only contain a subset of validdocuments might be selected by the query planner if no explicit index wasspecified at query time. Further, if a sort field is specified at query time,that field needs to exist (but could be null) in the results returned.
16.3.3.5. Other
The 2.1.1 release also includes the following minor improvements:
- #635: Stop couchindex processes on ddoc update- #721: Save migrated replicator checkpoint documents immediately- #688: Reuse http-based replication checkpoints when upgradingto https- #729: Recommend the use only of-name
and not-sname
in_vm.args for compatibility.- #738: Allow replicator application to always update replicatordocs.- #605: AddPrefer: return=minimal
header options fromRFC7240 to reduce the number of headers in the response.- #744: Allow a 503 response to be returned to clients (withmetric support)- #746: Log additional information on crashes from rexi- #752: Allow Mango $in queries without requiring the index touse an array- (multiple) Additional debugging utilities have been added.- (multiple) Hot code upgrades from 2.0 -> 2.1.1 are now possible.- (multiple) Improvements to the test suite have been made.- #765: Mangoexplain
now includes view parameters as requestedby the user.- #653: __show and _list should now work for admin-onlydatabases such as_users
.- #807: Mango index selection should occur only once.- #804: Unhandled Mango errors are now logged.- #659: Improve accuracy of themax_document_size
check.- #817: Invalid Base64 in inline attachments is now caught.- #825: Replication IDs no longer need to be URL encoded whenusing the_scheduler/jobs/<job_id>
endpoint.- #838: Do not buffer rexi messages to disconnected nodes.- #830: The stats collection interval is now configurable inan ini file, not in the application context. The default value is 10,and the setting is reloaded every 600 seconds.- #812: The/{db}
endpoint now includes acluster
blockwith the database’sq
,n
, and defaultw
andr
values.This supplements the existing/{db}/_shards
and/{db}/_shards/{id}
detailed information on sharding and quorum.- #810: The replicator scheduler crashed counter gauge morereliably detects replication crashes by reducing the default numberof retries from 10 to 5 (reducing the duration from 4 mins to 8 secs).- COUCHDB-3288: Tolerate mixed clusters for the upcoming pluggablestorage engine work.- #839: Mango python tests now support Python 3 as well as 2.- #845: A convenienceremsh
script has been added to supportlive debugging of running systems.- #846: Replicator logging is now less verbose and more informativewhen replication terminates unexpectedly.- #797: Reduce overflow errors are now returned to the client,allowing views with a single bad reduce to build while not exhausting theserver’s RAM usage.- #881: Mango now allows match on documents where the indexedvalue is an object if a range query is issued. Previously, query resultsmight change in the presence of an index, and operators/selectors whichexplicitly depend on a full index scan (such as$exists
) would notreturn a complete result set.- #883: Erlang time module compatibility has been improved forreleases of Erlang newer than 18.0.- #933: 410 is now returned when attempting to make a temporaryview request.- #934: The replicator now has a configurable delay beforeretrying to retrieve a document after receiving amissing_doc
error.- #936: jiffy now deduplicates JSON keys.
16.3.4. Version 2.1.0
The Mango
_find
endpoint supports a new combination operator,$allMatch
, which matches and returns all documents that contain anarray field with all its elements matching all the specified querycriteria.New scheduling replicator. The core of the new replicator is ascheduler which allows running a large number of replicationjobs by switching between them, stopping some and starting othersperiodically. Jobs which fail are backed off exponentially. There isalso an improved inspection and querying API:
_scheduler/jobs
and_scheduler/docs
:_scheduler/jobs
: This endpoint shows active replicationjobs. These are jobs managed by the scheduler. Some of them mightbe running, some might be waiting to run, or backed off(penalized) because they crashed too many times. Semantically thisis somewhat equivalent to_active_tasks
but focuses only onreplications. Jobs which have completed or which were nevercreated because of malformed replication documents will not beshown here as they are not managed by the scheduler._replicate
replications, started form _replicate endpoint notfrom a document in a_replicator
db, will also show up here._scheduler/docs
: This endpoint is an improvement on having to goback and read replication documents to query their state. Itrepresents the state of all the replications started fromdocuments in _replicator db. Unlike_scheduler/jobs
it will alsoshow jobs which have failed or have completed.
By default, scheduling replicator will not update documents withtransient states liketriggered
orerror
anymore, instead_scheduler/docs
API should be used to query replication documentstates.
16.3.4.1. Other scheduling replicator improvements
- Network resource usage and performance was improved byimplementing a shared connection pool. This should help in casesof a large number of connections to the same sources ortarget. Previously connection pools were shared only withing asingle replication job.- Improved request rate limit handling. Replicator requests willauto-discover rate limit capacity on targets and sources based ona proven Additive Increase / Multiplicative Decrease feedbackcontrol algorithm.- Improved performance by having exponential backoff for allreplication jobs failures. Previously there were some scenarioswere failure led to continuous repeated retries, consuming CPU anddisk resources in the process.- Improved recovery from long but temporary networkfailure. Currently if replications jobs fail to start 10 times ina row, they will not be retried anymore. This is sometimesdesirable, but in some cases, for example, after a sustained DNSfailure which eventually recovers, replications reach their retrylimit, stop retrying and never recover. Previously it requireduser intervention to continue. Scheduling replicator will nevergive up retrying a valid scheduled replication job and so itshould recover automatically.- Better handling of filtered replications. Failing user filter codefetches from the source will not block replicator manager andstall other replications. Failing filter fetches will also bebacked off exponentially. Another improvement is when filter codechanges on the source, a running replication will detect that andrestart itself with a new replication ID automatically.
The 2.1.0 release also includes the following minor improvements:
- COUCHDB-1946: Hibernate couch_stream after each write (up to 70% reductionin memory usage during replication of DBs with large attachments)- COUCHDB-2964: Investigate switching replicator manager change feeds tousing “normal” instead of “longpoll”- COUCHDB-2988: (mango) Allow query selector as changes and replicationfilter- COUCHDB-2992: Add additional support for document size- COUCHDB-3046: Improve reduce function overflow protection- COUCHDB-3061: Use vectored reads to search for buried headers in .couchfiles. “On a modern linux system with SSD, we see improvements up to 15x.”- COUCHDB-3063: “stale=ok” option replaced with new “stable” and “update”options.- COUCHDB-3180: Add features list in the welcome message- COUCHDB-3203: Make auth handlers configurable (in ini files)- COUCHDB-3234: Track open shard timeouts with a counter instead of logging- COUCHDB-3242: Make get view group info timeout in couch_indexerconfigurable- COUCHDB-3249: Add config to disable index all fields (text indexes)- COUCHDB-3251: Remove hot loop usage of filename:rootname/1- COUCHDB-3284: 8Kb read-ahead in couch_file causes extra IO and binarymemory usage- COUCHDB-3298: Optimize writing btree nodes- COUCHDB-3302: (Improve) Attachment replication over low bandwidth networkconnections- COUCHDB-3307: Limit calls to maybe_add_sys_db_callbacks to once per dbopen- COUCHDB-3318: bypass couch_httpd_vhost if there are none- COUCHDB-3323: Idle dbs cause excessive overhead- COUCHDB-3324: Introduce couch_replicator_scheduler- COUCHDB-3337: End-point _local_docs doesn’t conform to query params of_all_docs- COUCHDB-3358: (mango) Use efficient set storage for field names- COUCHDB-3425: Make _doc_ids _changes filter fast-path limit configurable- #457: TeX/LaTeX/texinfo removed from default docs build chain- #469: (mango) Choose index based on fields match- #483: couchup database migration tool- #582: Add X-Frame-Options support to help protect againstclickjacking- #593: Allow bind address of 127.0.0.1 in_cluster_setup
forsingle nodes- #624: Enable compaction daemon by default- #626: Allow enable node decom using string “true”- (mango) Configurable default limit, defaults to 25.- (mango) _design documents ignored when querying _all_docs- (mango) add $allMatch selector- Add local.d/default.d directories by default and document- Improved INSTALL.* text files
16.3.5. Fixed Issues
The 2.1.0 release includes fixes for the following issues:
- COUCHDB-1447: X-Couch-Update-NewRev header is missed if custom headers arespecified in response of _update handler (missed in 2.0 merge)
- COUCHDB-2731: Authentication DB was not considered a system DB
- COUCHDB-3010: (Superceded fix for replication exponential backoff)
- COUCHDB-3090: Error when handling empty “Access-Control-Request-Headers”header
- COUCHDB-3100: Fix documentation on require_valid_user
- COUCHDB-3109: 500 when include_docs=true for linked documents
- COUCHDB-3113: fabric:open_revs can return {ok, []}
- COUCHDB-3149: Exception written to the log if db deleted while there is achange feed running
- COUCHDB-3150: Update all shards with stale=update_after
- COUCHDB-3158: Fix a crash when connection closes for _update
- COUCHDB-3162: Default ssl settings cause a crash
- COUCHDB-3164: Request fails when using_changes?feed=eventsource&heartbeat=30000
- COUCHDB-3168: Replicator doesn’t handle well writing documents to a targetdb which has a small max_document_size
- COUCHDB-3173: Views return corrupt data for text fields containing non-BMPcharacters
- COUCHDB-3174: max_document_size setting can by bypassed by issuingmultipart/related requests
- COUCHDB-3178: Fabric does not send message when filtering lots of documents
- COUCHDB-3181: function_clause error when adding attachment to doc in _usersdb
- COUCHDB-3184: couch_mrview_compactor:recompact/1 does not handle errors inspawned process
- COUCHDB-3193: fabric:open_revs returns multiple results when one of theshards has stem_interactive_updates=false
- COUCHDB-3199: Replicator VDU function doesn’t acount for an alreadymalformed document in replicator db
- COUCHDB-3202: (mango) do not allow empty field names
- COUCHDB-3220: Handle timeout in _revs_diff
- COUCHDB-3222: (Fix) HTTP code 500 instead of 400 for invalid key duringdocument creation
- COUCHDB-3231: Allow fixing users’ documents (type and roles)
- COUCHDB-3232: user context not passed down in fabric_view_all_docs
- COUCHDB-3238: os_process_limit documentation wrong
- COUCHDB-3241: race condition in couch_server if delete msg for a db isreceived before open_result msg
- COUCHDB-3245: Make couchjs -S option take effect again
- COUCHDB-3252: Include main-coffee.js in release artifact (brokenCoffeeScript view server)
- COUCHDB-3255: Conflicts introduced by recreating docs with attachments
- COUCHDB-3259: Don’t trap exits in couch_file
- COUCHDB-3264: POST to _all_docs does not respect conflicts=true
- COUCHDB-3269: view response can ‘hang’ with filter and limit specified
- COUCHDB-3271: Replications crash with ‘kaboom’ exit
- COUCHDB-3274: eof in couch_file can be incorrect after error
- COUCHDB-3277: Replication manager crashes when it finds _replicator dbshards which are not part of a mem3 db
- COUCHDB-3286: Validation function throwing unexpected json crashes withfunction_clause
- COUCHDB-3289: handle error clause when calling fabric:open_revs
- COUCHDB-3291: Excessively long document IDs prevent replicator from makingprogress
- COUCHDB-3293: Allow limiting length of document ID (for CouchDB proper)
- COUCHDB-3305: (mango) don’t crash with invalid input to built in reducerfunction
- COUCHDB-3362: DELETE attachment on non-existing document creates thedocument, rather than returning 404
- COUCHDB-3364: Don’t crash compactor when compacting process fails.
- COUCHDB-3367: Require server admin user for db/_compact and db_view_cleanupendpoints
- COUCHDB-3376: Fix mem3_shards under load
- COUCHDB-3378: Fix mango full text detection
- COUCHDB-3379: Fix couch_auth_cache reinitialization logic
- COUCHDB-3400: Notify couch_index_processes on all shards when ddoc updated
- COUCHDB-3402: race condition in mem3 startup
- #511: (mango) Return false for empty list
- #595: Return 409 to PUT attachment with non-existent rev
- #623: Ensure replicator _active_tasks entry reports recent pendingchanges value
- #627: Pass UserCtx to fabric’s all_docs from mango query
- #631: fix couchdb_os_proc_pool eunit timeouts
- #644: Make couch_event_sup:stop/1 synchronous
- #645: Pass db open options to fabric_view_map for _view and _listqueries on _users DB
- #648: Fix couch_replicator_changes_reader:process_change
- #649: Avoid a race when restarting an index updater
- #667: Prevent a terrible race condition
- #677: Make replication filter fetch error for _replicate return a404
- Fix CORS
max_age
configuration parameter via Access-Control-Max-Age - Chunk missing revisions before attempting to save on target (improvesreplication for very conflicted, very deep revision tree documents)
- Allow w parameter for attachments
- Return “Bad Request” when count in
/_uuids
exceeds max - Fix crashes when replicator db is deleted
- Skip internal replication if changes already replicated
- Fix encoding issues on
_update/../doc_id
and PUT attachments