Incompatible changes in ArangoDB 3.0

It is recommended to check the following list of incompatible changes beforeupgrading to ArangoDB 3.0, and adjust any client programs if necessary.

Build system

Building ArangoDB 3.0 from source now requires CMake.

The pre-3.0 build system used a configure-based approach. The steps to buildArangoDB 2.8 from source code were:

  1. make setup
  2. ./configure <options>
  3. make

These steps will not work anymore, as ArangoDB 3.0 does not come with aconfigure script.

To build 3.0 on Linux, create a separate build directory first:

  1. mkdir -p build

and then create the initial build scripts once using CMake:

  1. (cd build && cmake <options> ..)

The above command will configure the build and check for the requireddependencies. If everything works well the actual build can be started with

  1. (cd build && make)

The binaries for the ArangoDB server and all client tools will then be createdinside the build directory. To start ArangoDB locally from the build directory,use

  1. build/bin/arangod <options>

Datafiles and datafile names

ArangoDB 3.0 uses a new VelocyPack-based format for storing data in WAL logfilesand collection datafiles. The file format is not compatible with the files usedin prior versions of ArangoDB. That means datafiles written by ArangoDB 3.0 cannot beused in earlier versions and vice versa.

The pattern for collection directory names was changed in 3.0 to include a random id component at the end. The new pattern is collection-<id>-<random>, where <id> is the collection id and <random> is a random number. Previous versions of ArangoDB used a pattern collection-<id> without the random number.

User Management

Unlike ArangoDB 2.x, ArangoDB 3.0 users are now separated from databases, and you can grant one or more database permissions to a user.

If you want to mimic the behavior of ArangoDB, you should name your users like username@dbname.

Users that can access the _system database are allowed to manage users andpermissions for all databases.

Edges and edges attributes

In ArangoDB prior to 3.0 the attributes _from and _to of edges were treatedspecially when loading or storing edges. That special handling led to these attributesbeing not as flexible as regular document attributes. For example, the _from and _to attribute values of an existing edge could not be updated once the edge wascreated. Additionally, the _from and _to attributes could not be indexed inuser-defined indexes, e.g. to make each combination of _from and _to unique.Finally, as _from and _to referenced the linked collections by collection idand not by collection name, their meaning became unclear once a referenced collection was dropped. The collection id stored in edges then became unusable, and whenaccessing such edge the collection name part of it was always translated to _undefined.

In ArangoDB 3.0, the _from and _to values of edges are saved as regular strings.This allows using _from and _to in user-defined indexes. Additionally this allowsupdating the _from and _to values of existing edges. Furthermore, collectionsreferenced by _from and _to values may be dropped and re-created later. Any_from and _to values of edges pointing to such dropped collection are unaffectedby the drop operation now. Also note that renaming the collection referenced in_from and _to in ArangoDB 2.8 also relinked the edges. In 3.0 the edges are NOTautomatically relinked to the new collection anymore.

Documents

Documents (in contrast to edges) cannot contain the attributes _from or _to on themain level in ArangoDB 3.0. These attributes will be automatically removed when savingdocuments (i.e. non-edges). _from and _to can be still used in sub-objects insidedocuments.

The _from and _to attributes will of course be preserved and are still required when saving edges.

AQL

Edges handling

When updating or replacing edges via AQL, any modifications to the _from and _toattributes of edges were ignored by previous versions of ArangoDB, without signalingany errors. This was due to the _from and _to attributes being immutable in earlierversions of ArangoDB.

From 3.0 on, the _from and _to attributes of edges are mutable, so any AQL queries thatmodify the _from or _to attribute values of edges will attempt to actually change theseattributes. Clients should be aware of this change and should review their queries thatmodify edges to rule out unintended side-effects.

Additionally, when completely replacing the data of existing edges via the AQL REPLACEoperation, it is now required to specify values for the _from and _to attributes,as REPLACE requires the entire new document to be specified. If either _from or _toare missing from the replacement document, an REPLACE operation will fail.

Graph functions

In version 3.0 all former graph related functions have been removed from AQL tobe replaced by native AQL constructs.These constructs allow for more fine-grained filtering on several graph levels.Also this allows the AQL optimizer to automatically improve these queries byenhancing them with appropriate indexes.We have created recipes to upgrade from 2.8 to 3.0 when using these functions.

The functions:

Typecasting functions

The type casting applied by the TO_NUMBER() AQL function has changed as follows:

  • string values that do not contain a valid numeric value are now converted to the number0. In previous versions of ArangoDB such string values were converted to the valuenull.
  • array values with more than 1 member are now converted to the number 0. In previousversions of ArangoDB such arrays were converted to the value null.
  • objects / documents are now converted to the number 0. In previous versions of ArangoDBobjects / documents were converted to the value null.Additionally, the TO_STRING() AQL function now converts null values into an empty string("") instead of the string "null", which is more in line with LENGTH(null) returning0 and not 4 since v2.6.

The output of TO_STRING() has also changed for arrays and objects as follows:

  • arrays are now converted into their JSON-stringify equivalents, e.g.

    • [ ] is now converted to []
    • [ 1, 2, 3 ] is now converted to [1,2,3]

    • [ "test", 1, 2 ] is now converted to [“test”,1,2]`Previous versions of ArangoDB converted arrays with no members into theempty string, and non-empty arrays into a comma-separated list of membervalues, without the surrounding angular brackets. Additionally, stringarray members were not enclosed in quotes in the result string:

    • [ ] was converted to ``

    • [ 1, 2, 3 ] was converted to 1,2,3
    • [ "test", 1, 2 ] was converted to test,1,2`

  • objects are now converted to their JSON-stringify equivalents, e.g.

    • { } is converted to {}
    • { a: 1, b: 2 } is converted to {"a":1,"b":2}
    • { "test" : "foobar" } is converted to {"test":"foobar"}Previous versions of ArangoDB always converted objects into the string[object Object]

This change also affects other parts in AQL that used TO_STRING() to implicitlycast operands to strings. It also affects the AQL functions CONCAT() and CONCAT_SEPARATOR() which treated array values differently. Previous versionsof ArangoDB automatically flattened array values in the first level of the array, e.g. CONCAT([1, 2, 3, [ 4, 5, 6 ]]) produced 1,2,3,4,5,6. Now this will produce[1,2,3,[4,5,6]]. To flatten array members on the top level, you can now usethe more explicit CONCAT(FLATTEN([1, 2, 3, [4, 5, 6]], 1)).

Arithmetic operators

As the arithmetic operations in AQL implicitly convert their operands to numeric values using TO_NUMBER(), their casting behavior has also changed as described above.

Some examples of the changed behavior:

  • "foo" + 1 produces 1 now. In previous versions this produced null.
  • [ 1, 2 ] + 1 produces 1. In previous versions this produced null.
  • 1 + "foo" + 1´ produces 2 now. In previous version this produced 1`.

Attribute names and parameters

Previous versions of ArangoDB had some trouble with attribute names that contained the dotsymbol (.). Some code parts in AQL used the dot symbol to split an attribute name intosub-components, so an attribute named a.b was not completely distinguishable from anattribute a with a sub-attribute b. This inconsistent behavior sometimes allowed “hacks”to work such as passing sub-attributes in a bind parameter as follows:

  1. FOR doc IN collection
  2.   FILTER doc.@name == 1
  3.   RETURN doc

If the bind parameter @name contained the dot symbol (e.g. @bind = a.b, it was unclear whether this should trigger sub-attribute access (i.e. doc.a.b) or a access to an attribute with exactly the specified name (i.e. doc["a.b"]).

ArangoDB 3.0 now handles attribute names containing the dot symbol properly, and sending abind parameter @name = a.b will now always trigger an access to the attribute doc["a.b"],not the sub-attribute b of a in doc.

For users that used the “hack” of passing bind parameters containing dot symbol to accesssub-attributes, ArangoDB 3.0 allows specifying the attribute name parts as an array of strings,e.g. @name = [ "a", "b" ], which will be resolved to the sub-attribute access doc.a.b when the query is executed.

Keywords

LIKE is now a keyword in AQL. Using LIKE in either case as an attribute or collection name in AQL queries now requires quoting.

SHORTEST_PATH is now a keyword in AQL. Using SHORTEST_PATH in either case as an attribute or collection name in AQL queries now requires quoting.

Subqueries

Queries that contain subqueries that contain data-modification operations such as INSERT, UPDATE, REPLACE, UPSERT or REMOVE will now refuse to execute if the collectionaffected by the subquery’s data-modification operation is read-accessed in an outer scope of the query.

For example, the following query will refuse to execute as the collection myCollectionis modified in the subquery but also read-accessed in the outer scope:

  1. FOR doc IN myCollection
  2.   LET changes = (
  3.     FOR what IN myCollection
  4.       FILTER what.value == 1
  5.       REMOVE what IN myCollection
  6.   )
  7.   RETURN doc

It is still possible to write to collections from which data is read in the same query,e.g.

  1. FOR doc IN myCollection
  2.   FILTER doc.value == 1
  3.   REMOVE doc IN myCollection

and to modify data in different collection via subqueries.

Other changes

The AQL optimizer rule “merge-traversal-filter” that already existed in 3.0 was renamed to “optimize-traversals”. This should be of no relevance to client applications except if they programatically look for applied optimizer rules in the explain out of AQL queries.

The order of results created by the AQL functions VALUES() and ATTRIBUTES() was never guaranteed and it only had the “correct” ordering by accident when iterating over objects that were not loaded from the database. As some of the function internals have changed, the“correct” ordering will not appear anymore, and still no result order is guaranteed bythese functions unless the sort parameter is specified (for the ATTRIBUTES() function).

Upgraded V8 version

The V8 engine that is used inside ArangoDB to execute JavaScript code has been upgraded fromversion 4.3.61 to 5.0.71.39. The new version should be mostly compatible to the old version,but there may be subtle differences, including changes of error message texts thrown by theengine.Furthermore, some V8 startup parameters have changed their meaning or have been removed inthe new version. This is only relevant when ArangoDB or ArangoShell are started with a customvalue for the —javascript.v8-options startup option.

Among others, the following V8 options change in the new version of ArangoDB:

  • —es_staging: in 2.8 it had the meaning enable all completed harmony features, in 3.0 the option means enable test-worthy harmony features (for internal use only)

  • —strong_this: this option wasn’t present in 2.8. In 3.0 it means don't allow 'this' toescape from constructors and defaults to true.

  • —harmony_regexps: this options means enable "harmony regular expression extensions" and changes its default value from false to true

  • —harmony_proxies: this options means enable "harmony proxies" and changes its default value from false to true

  • —harmony_reflect: this options means enable "harmony Reflect API" and changes its default value from false to true

  • —harmony_sloppy: this options means enable "harmony features in sloppy mode" and changes its default value from false to true

  • —harmony_tostring: this options means enable "harmony toString" and changes its default value from false to true

  • —harmony_unicode_regexps: this options means enable "harmony unicode regexps" and changes its default value from false to true

  • —harmony_arrays, —harmony_array_includes, —harmony_computed_property_names, —harmony_arrow_functions, —harmony_rest_parameters, —harmony_classes, —harmony_object_literals, —harmony_numeric_literals, —harmony_unicode: these option have been removed in V8 5.

As a consequence of the upgrade to V8 version 5, the implementation of theJavaScript Buffer object had to be changed. JavaScript Buffer objects inArangoDB now always store their data on the heap. There is no shared poolfor small Buffer values, and no pointing into existing Buffer data whenextracting slices. This change may increase the cost of creating Buffers withshort contents or when peeking into existing Buffers, but was required forsafer memory management and to prevent leaks.

JavaScript API changes

The following incompatible changes have been made to the JavaScript API in ArangoDB 3.0:

Foxx

The Foxx framework has been completely rewritten for 3.0 with a new, simpler and more familiar API. To make Foxx services developed for 2.8 or earlier ArangoDB versions run in 3.0, the service’s manifest file needs to be edited.

To enable the legacy mode for a Foxx service, add "engines": {"arangodb": "^2.8.0"} (or similar version ranges that exclude 3.0 and up) to the service manifest file(named “manifest.json”, located in the service’s base directory).

Require

Modules shipped with ArangoDB can now be required using the pattern @arangodb/<module>instead of org/arangodb/<module>, e.g.

  1. var cluster = require("@arangodb/cluster");

The old format can still be used for compatibility:

  1. var cluster = require("org/arangodb/cluster");

ArangoDB prior to version 3.0 allowed a transparent use of CoffeeScriptsource files with the require() function. Files with a file name extensionof coffee were automatically sent through a CoffeeScript parser andtranspiled into JavaScript on-the-fly. This support is gone with ArangoDB3.0. To run any CoffeeScript source files, they must be converted to JavaScriptby the client application.

Response object

The @arangodb/request response object now stores the parsed JSON responsebody in a property json instead of body when the request was made using thejson option. The body instead contains the response body as a string.

JavaScript Edges API

When completely replacing an edge via a collection’s replace() function the replacing edge data now needs to contain the _from and _to attributes for the new edge. Previousversions of ArangoDB did not require the edge data to contain _from and _to attributeswhen replacing an edge, since _from and _to values were immutable for existing edges.

For example, the following call worked in ArangoDB 2.8 but will fail in 3.0:

  1. db.edgeCollection.replace("myKey", { value: "test" });

To make this work in ArangoDB 3.0, _from and _to need to be added to the replacementdata:

  1. db.edgeCollection.replace("myKey", { _from: "myVertexCollection/1", _to: "myVertexCollection/2", value: "test" });

Note that this only affects the replace() function but not update(), which willonly update the specified attributes of the edge and leave all others intact.

Additionally, the functions edges(), outEdges() and inEdges() with an array of edge ids will now make the edge ids unique before returning the connected edges. This is probably desired anyway, as results will be returned only once per distinct input edge id. However, it may break client applications that rely on the old behavior.

Databases API

The _listDatabases() function of the db object has been renamed to _databases(), making it consistent with the _collections() function. Also the _listEndpoints() function has been renamed to _endpoints().

Collection API

Example matching

The collection function byExampleHash() and byExampleSkiplist() have been removed in 3.0.Their functionality is provided by collection’s byExample() function, which will automaticallyuse a suitable index if present.

The collection function byConditionSkiplist() has been removed in 3.0. The same functionalitycan be achieved by issuing an AQL query with the target condition, which will automatically usea suitable index if present.

Javascript Revision id handling

The exists() method of a collection now throws an exception when the specified documentexists but its revision id does not match the revision id specified. Previous versions ofArangoDB simply returned false if either no document existed with the specified key orwhen the revision id did not match. It was therefore impossible to distinguish these twocases from the return value alone. 3.0 corrects this. Additionally, exists() in previousversions always returned a boolean if only the document key was given. 3.0 now returns thedocument’s meta-data, which includes the document’s current revision id.

Given there is a document with key test in collection myCollection, then the behaviorof 3.0 is as follows:

  1. /* test if document exists. this returned true in 2.8 */
  2. db.myCollection.exists("test");
  3. {
  4.   "_key" : "test",
  5.   "_id" : "myCollection/test",
  6.   "_rev" : "9758059"
  7. }
  9. /* test if document exists. this returned true in 2.8 */
  10. db.myCollection.exists({ _key: "test" });
  11. {
  12.   "_key" : "test",
  13.   "_id" : "myCollection/test",
  14.   "_rev" : "9758059"
  15. }
  17. /* test if document exists. this also returned false in 2.8 */
  18. db.myCollection.exists("foo");
  19. false
  21. /* test if document with a given revision id exists. this returned true in 2.8 */
  22. db.myCollection.exists({ _key: "test", _rev: "9758059" });
  23. {
  24.   "_key" : "test",
  25.   "_id" : "myCollection/test",
  26.   "_rev" : "9758059"
  27. }
  29. /* test if document with a given revision id exists. this returned false in 2.8 */
  30. db.myCollection.exists({ _key: "test", _rev: "1234" });
  31. JavaScript exception: ArangoError 1200: conflict

Cap constraints

The cap constraints feature has been removed. This change has led to the removal of thecollection operations first() and last(), which were internally based on data from cap constraints.

As cap constraints have been removed in ArangoDB 3.0 it is not possible to create anindex of type “cap” with a collection’s ensureIndex() function. The dedicated functionensureCapConstraint() has also been removed from the collection API.

Graph Blueprints JS Module

The deprecated module graph-blueprints has been deleted.All it’s features are covered by the general-graph module.

General Graph Fluent AQL interface

The fluent interface has been removed from ArangoDB.It’s features were completely overlapping with “aqb”which comes pre installed as well.Please switch to AQB instead.

Undocumented APIs

The undocumented functions BY_EXAMPLE_HASH() and BY_EXAMPLE_SKIPLIST(),BY_CONDITION_SKIPLIST, CPP_NEIGHBORS and CPP_SHORTEST_PATH have been removed.These functions were always hidden and not intended to be part ofthe public JavaScript API for collections.

HTTP API changes

CRUD operations

The following incompatible changes have been made to the HTTP API in ArangoDB 3.0:

General

The HTTP insert operations for single documents and edges (POST /_api/document) do not support the URL parameter “createCollection” anymore. In previous versions ofArangoDB this parameter could be used to automatically create a collection uponinsertion of the first document. It is now required that the target collection alreadyexists when using this API, otherwise it will return an HTTP 404 error.The same is true for the import API at POST /_api/import.

Collections can still be created easily via a separate call to POST /_api/collectionas before.

The “location” HTTP header returned by ArangoDB when inserting a new document or edgenow always contains the database name. This was also the default behavior in previousversions of ArangoDB, but it could be overridden by clients sending the HTTP headerx-arango-version: 1.4 in the request. Clients can continue to send this header toArangoDB 3.0, but the header will not influence the location response headers produced by ArangoDB 3.0 anymore.

Additionally the CRUD operations APIs do not return an attribute “error” in theresponse body with an attribute value of “false” in case an operation succeeded.

Revision id handling

The operations for updating, replacing and removing documents can optionally check therevision number of the document to be updated, replaced or removed so the caller canensure the operation works on a specific version of the document and there are nolost updates.

Previous versions of ArangoDB allowed passing the revision id of the previous documenteither in the HTTP header If-Match or in the URL parameter rev. For example, removing a document with a specific revision id could be achieved as follows:

  1. curl -X DELETE \
  2.      "http://127.0.0.1:8529/_api/document/myCollection/myKey?rev=123"

ArangoDB 3.0 does not support passing the revision id via the “rev” URL parameteranymore. Instead the previous revision id must be passed in the HTTP header If-Match,e.g.

  1. curl -X DELETE \
  2.      --header "If-Match: '123'" \
  3.      "http://127.0.0.1:8529/_api/document/myCollection/myKey"

The URL parameter “policy” was also usable in previous versions of ArangoDB tocontrol revision handling. Using it was redundant to specifying the expected revisionid via the “rev” parameter or “If-Match” HTTP header and therefore support for the “policy”parameter was removed in 3.0.

In order to check for a previous revision id when updating, replacing or removing documents please use the If-Match HTTP header as described above. When no revisioncheck if required the HTTP header can be omitted, and the operations will work on thecurrent revision of the document, regardless of its revision id.

All documents API

The HTTP API for retrieving the ids, keys or URLs of all documents from a collectionwas previously located at GET /_api/document?collection=…. This API was moved toPUT /_api/simple/all-keys and is now executed as an AQL query.The name of the collection must now be passed in the HTTP request body instead of inthe request URL. The same is true for the “type” parameter, which controls the type ofthe result to be created.

Calls to the previous API can be translated as follows:

  • old: GET /_api/document?collection=<collection>&type=<type> without HTTP request body
  • 3.0: PUT /_api/simple/all-keys with HTTP request body {"collection":"<collection>","type":"id"}The result format of this API has also changed slightly. In previous versions calls tothe API returned a JSON object with a documents attribute. As the functionality isbased on AQL internally in 3.0, the API now returns a JSON object with a result attribute.

Edges API

CRUD operations on edges

The API for documents and edges have been unified in ArangoDB 3.0. The CRUD operations for documents and edges are now handled by the same endpoint at /_api/document. For CRUD operations there is no distinction anymore between documents and edges API-wise.

That means CRUD operations concerning edges need to be sent to the HTTP endpoint /_api/document instead of /_api/edge. Sending requests to /_api/edge willresult in an HTTP 404 error in 3.0. The following methods are available at/_api/document for documents and edge:

  • HTTP POST: insert new document or edge
  • HTTP GET: fetch an existing document or edge
  • HTTP PUT: replace an existing document or edge
  • HTTP PATCH: partially update an existing document or edge
  • HTTP DELETE: remove an existing document or edgeWhen completely replacing an edge via HTTP PUT please note that the replacing edgedata now needs to contain the _from and _to attributes for the edge. Previousversions of ArangoDB did not require sending _from and _to when replacing edges, as _from and _to values were immutable for existing edges.

The _from and _to attributes of edges now also need to be present inside theedges objects sent to the server:

  1. curl -X POST \
  2.      --data '{"value":1,"_from":"myVertexCollection/1","_to":"myVertexCollection/2"}' \
  3.      "http://127.0.0.1:8529/_api/document?collection=myEdgeCollection"

Previous versions of ArangoDB required the _from and _to attributes of edges be sent separately in URL parameter from and to:

  1. curl -X POST \
  2.      --data '{"value":1}' \
  3.      "http://127.0.0.1:8529/_api/edge?collection=e&from=myVertexCollection/1&to=myVertexCollection/2"

Querying connected edges

The REST API for querying connected edges at GET /_api/edges/<collection> will nowmake the edge ids unique before returning the connected edges. This is probably desired anywayas results will now be returned only once per distinct input edge id. However, it may break client applications that rely on the old behavior.

Graph API

Some data-modification operations in the named graphs API at /_api/gharial now return eitherHTTP 202 (Accepted) or HTTP 201 (Created) if the operation succeeds. Which status code is returneddepends on the waitForSync attribute of the affected collection. In previous versions someof these operations return HTTP 200 regardless of the waitForSync value.

The deprecated graph API /_api/graph has been removed.All it’s features can be replaced using /_api/gharial and AQL instead.

Simple queries API

The REST routes PUT /_api/simple/first and /_api/simple/last have been removedentirely. These APIs were responsible for returning the first-inserted andlast-inserted documents in a collection. This feature was built on cap constraintsinternally, which have been removed in 3.0.

Calling one of these endpoints in 3.0 will result in an HTTP 404 error.

Indexes API

It is not supported in 3.0 to create an index with type cap (cap constraint) in 3.0 as the cap constraints feature has bee removed. Calling the index creationendpoint HTTP API POST /_api/index?collection=… with an index type cap will therefore result in an HTTP 400 error.

Log entries API

The REST route HTTP GET /_admin/log is now accessible from within all databases. Inprevious versions of ArangoDB, this route was accessible from within the _systemdatabase only, and an HTTP 403 (Forbidden) was thrown by the server for any accessfrom within another database.

Figures API

The REST route HTTP GET /_api/collection/<collection>/figures will not return the following result attributes as they became meaningless in 3.0:

  • shapefiles.count
  • shapes.fileSize
  • shapes.count
  • shapes.size
  • attributes.count
  • attributes.size

Databases and Collections APIs

When creating a database via the API POST /_api/database, ArangoDB will now alwaysreturn the HTTP status code 202 (created) if the operation succeeds. Previous versionsof ArangoDB returned HTTP 202 as well, but this behavior was changable by sending anHTTP header x-arango-version: 1.4. When sending this header, previous versions ofArangoDB returned an HTTP status code 200 (ok). Clients can still send this header toArangoDB 3.0 but this will not influence the HTTP status code produced by ArangoDB.

The “location” header produced by ArangoDB 3.0 will now always contain the database name. This was also the default in previous versions of ArangoDB, but the behaviorcould be overridden by sending the HTTP header x-arango-version: 1.4. Clients canstill send the header, but this will not make the database name in the “location”response header disappear.

The result format for querying all collections via the API GET /_api/collection has been changed.

Previous versions of ArangoDB returned an object with an attribute named collections and an attribute named names. Both contained all available collections, butcollections contained the collections as an array, and names contained thecollections again, contained in an object in which the attribute names were thecollection names, e.g.

  1. {
  2.   "collections": [
  3.     {"id":"5874437","name":"test","isSystem":false,"status":3,"type":2},
  4.     {"id":"17343237","name":"something","isSystem":false,"status":3,"type":2},
  5.     ...
  6.   ],
  7.   "names": {
  8.     "test": {"id":"5874437","name":"test","isSystem":false,"status":3,"type":2},
  9.     "something": {"id":"17343237","name":"something","isSystem":false,"status":3,"type":2},
  10.     ...
  11.   }
  12. }

This result structure was redundant, and therefore has been simplified to just

  1. {
  2.   "result": [
  3.     {"id":"5874437","name":"test","isSystem":false,"status":3,"type":2},
  4.     {"id":"17343237","name":"something","isSystem":false,"status":3,"type":2},
  5.     ...
  6.   ]
  7. }

in ArangoDB 3.0.

Replication APIs

The URL parameter “failOnUnknown” was removed from the REST API GET /_api/replication/dump.This parameter controlled whether dumping or replicating edges should fail if oneof the vertex collections linked in the edge’s _from or _to attributes was notpresent anymore. In this case the _from and _to values could not be translated intomeaningful ids anymore.

There were two ways for handling this:

  • setting failOnUnknown to true caused the HTTP request to fail, leaving error handling to the user
  • setting failOnUnknown to false caused the HTTP request to continue, translatingthe collection name part in the _from or _to value to _unknown.In ArangoDB 3.0 this parameter is obsolete, as _from and _to are stored as self-containedstring values all the time, so they cannot get invalid when referenced collections aredropped.

The result format of the API GET /_api/replication/logger-follow has changed slightly inthe following aspects:

  • documents and edges are reported in the same way. The type for document insertions/updatesand edge insertions/updates is now always 2300. Previous versions of ArangoDB returneda type value of 2300 for documents and 2301 for edges.
  • records about insertions, updates or removals of documents and edges do not have thekey and rev attributes on the top-level anymore. Instead, key and rev can be accessed by peeking into the _key and _rev attributes of the data sub-attributesof the change record.The same is true for the collection-specific changes API GET /_api/replication/dump.

User management APIs

The REST API endpoint POST /_api/user for adding new users now requires the request tocontain a JSON object with an attribute named user, containing the name of the user tobe created. Previous versions of ArangoDB also checked this attribute, but additionally looked for an attribute username if the user attribute did not exist.

Undocumented HTTP APIs

The following undocumented HTTP REST endpoints have been removed from ArangoDB’s RESTAPI:

  • /_open/cerberus and /_system/cerberus: these endpoints were intended for some ArangoDB-internal applications only
  • PUT /_api/simple/by-example-hash, PUT /_api/simple/by-example-skiplist andPUT /_api/simple/by-condition-skiplist: these methods were documented in earlyversions of ArangoDB but have been marked as not intended to be called by endusers since ArangoDB version 2.3. These methods should not have been part of anyArangoDB manual since version 2.4.
  • /_api/structure: an older unfinished and unpromoted API for data format and type checks, superseded by Foxx applications.

Administration APIs

  • /_admin/shutdown now needs to be called with the HTTP DELETE method

Handling of CORS requests

It can now be controlled in detail for which origin hosts CORS (Cross-origin resource sharing) requests with credentials will be allowed. ArangoDB 3.0 provides the startupoption —http.trusted-origin that can be used to specify one or many origins fromwhich CORS requests are treated as “trustworthy”.

The option can be specified multiple times, once per trusted origin, e.g.

  1. --http.trusted-origin http://127.0.0.1:8529 --http.trusted-origin https://127.0.0.1:8599

This will make the ArangoDB server respond to CORS requests from these origins with anAccess-Control-Allow-Credentials HTTP header with a value of true. Web browsers caninspect this header and can allow passing ArangoDB web interface credentials (if stored in the browser) to the requesting site. ArangoDB will not forward or provide any credentials.

Setting this option is only required if applications on other hosts need to access the ArangoDB web interface or other HTTP REST APIs from a web browser with the same credentials that the user has entered when logging into the web interface. When a web browser finds the Access-Control-Allow-Credentials HTTP response header, it may forward the credentialsentered into the browser for the ArangoDB web interface login to the other site.

This is a potential security issue, so there are no trusted origins by default. It maybe required to set some trusted origins if you’re planning to issue AJAX requests to ArangoDBfrom other sites from the browser, with the credentials entered during the ArangoDB interface login (i.e. single sign-on). If such functionality is not used, the option should notbe set.

To specify a trusted origin, specify the option once per trusted origin as shown above.Note that the trusted origin values specified in this option will be compared bytewisewith the Origin HTTP header value sent by clients, and only exact matches will pass.

There is also the wildcard all for enabling CORS access from all origins in a test or development setup:

  1. --http.trusted-origin all

Setting this option will lead to the ArangoDB server responding with an Access-Control-Allow-Credentials: true HTTP header to all incoming CORS requests.

Command-line options

Quite a few startup options in ArangoDB 2 were double negations (like—server.disable-authentication false). In ArangoDB 3 these are now expressed as positives (e. g. —server.authentication). Also the options between the ArangoDBserver and its client tools have being unified. For example, the logger options are now the same for the server and the client tools. Additionally many options havebeen moved into more appropriate topic sections.

Renamed options

The following options have been available before 3.0 and have changed their name in 3.0:

  • —server.disable-authentication was renamed to —server.authentication.Note that the meaning of the option —server.authentication is the opposite of the previous —server.disable-authentication.
  • —server.disable-authentication-unix-sockets was renamed to—server.authentication-unix-sockets. Note that the meaning of the option—server.authentication-unix-sockets is the opposite of the previous—server.disable-authentication-unix-sockets.
  • —server.authenticate-system-only was renamed to —server.authentication-system-only.The meaning of the option in unchanged.
  • —server.disable-statistics was renamed to —server.statistics. Note that themeaning of the option —server.statistics is the opposite of the previous—server.disable-statistics.
  • —server.cafile was renamed to —ssl.cafile. The meaning of the option isunchanged.
  • —server.keyfile was renamed to —ssl.keyfile. The meaning of the option isunchanged.
  • —server.ssl-cache was renamed to —ssl.session-cache. The meaning of the optionis unchanged.
  • —server.ssl-cipher-list was renamed to —ssl.cipher-list. The meaning of theoption is unchanged.
  • —server.ssl-options was renamed to —ssl.options. The meaning of the optionis unchanged.
  • —server.ssl-protocol was renamed to —ssl.protocol. The meaning of the optionis unchanged.
  • —server.backlog-size was renamed to —tcp.backlog-size. The meaning of theoption is unchanged.
  • —server.reuse-address was renamed to —tcp.reuse-address. The meaning of theoption is unchanged.
  • —server.disable-replication-applier was renamed to —database.replication-applier.The meaning of the option —database.replication-applier is the opposite of theprevious —server.disable-replication-applier.
  • —server.allow-method-override was renamed to —http.allow-method-override. Themeaning of the option is unchanged.
  • —server.hide-product-header was renamed to —http.hide-product-header. Themeaning of the option is unchanged.
  • —server.keep-alive-timeout was renamed to —http.keep-alive-timeout. Themeaning of the option is unchanged.
  • —server.foxx-queues was renamed to —foxx.queues. The meaning of the optionis unchanged.
  • —server.foxx-queues-poll-interval was renamed to —foxx.queues-poll-interval.The meaning of the option is unchanged.
  • —no-server was renamed to —server.rest-server. Note that the meaning of theoption —server.rest-server is the opposite of the previous —no-server.
  • —database.query-cache-mode was renamed to —query.cache-mode. The meaning ofthe option is unchanged.
  • —database.query-cache-max-results was renamed to —query.cache-entries. Themeaning of the option is unchanged.
  • —database.disable-query-tracking was renamed to —query.tracking. The meaning of the option —query.tracking is the opposite of the previous—database.disable-query-tracking.
  • —log.tty was renamed to —log.foreground-tty. The meaning of the option isunchanged.
  • —upgrade has been renamed to —database.auto-upgrade. In contrast to 2.8 thisoption now requires a boolean parameter. To actually perform an automatic database upgrade at startup use —database.auto-upgrade true. To not perform it, use—database.auto-upgrade false.
  • —check-version has been renamed to —database.check-version.
  • —temp-path has been renamed to —temp.path.

Log verbosity, topics and output files

Logging now supports log topics. You can control these by specifying a logtopic in front of a log level or an output. For example

  1.   --log.level startup=trace --log.level info

will log messages concerning startup at trace level, everything else at infolevel. —log.level can be specified multiple times at startup, for as manytopics as needed.

Some relevant log topics available in 3.0 are:

  • collector: information about the WAL collector’s state
  • compactor: information about the collection datafile compactor
  • datafiles: datafile-related operations
  • mmap: information about memory-mapping operations
  • performance: some performance-related information
  • queries: executed AQL queries
  • replication: replication-related info
  • requests: HTTP requests
  • startup: information about server startup and shutdown

  • threads: information about threadsThe new log option —log.output <definition> allows directing the globalor per-topic log output to different outputs. The output definition “" can be one of

  • ”-“ for stdin

  • ”+” for stderr
  • “syslog://"
  • “syslog:///"
  • “file://"The option can be specified multiple times in order to configure the outputfor different log topics. To set up a per-topic output configuration, use—log.output <topic>=<definition>, e.g.

queries=file://queries.txt

logs all queries to the file “queries.txt”.

The old option —log.file is still available in 3.0 for convenience reasons. In3.0 it is a shortcut for the more general option —log.output file://filename.

The old option —log.requests-file is still available in 3.0. It is now a shortcutfor the more general option —log.output requests=file://….

The old option —log.performance is still available in 3.0. It is now a shortcutfor the more general option —log.level performance=trace.

Removed options for logging

The options —log.content-filter and —log.source-filter have been removed. Theyhave most been used during ArangoDB’s internal development.

The syslog-related options —log.application and —log.facility have been removed.They are superseded by the more general —log.output option which can also handle syslog targets.

Removed other options

The option —server.default-api-compatibility was present in earlier version ofArangoDB to control various aspects of the server behavior, e.g. HTTP return codesor the format of HTTP “location” headers. Client applications could send an HTTPheader “x-arango-version” with a version number to request the server behavior ofa certain ArangoDB version.

This option was only honored in a handful of cases (described above) and was removedin 3.0 because the changes in server behavior controlled by this option were changedeven before ArangoDB 2.0. This should have left enough time for client applicationsto adapt to the new behavior, making the option superfluous in 3.0.

Thread options

The options —server.threads and —scheduler.threads now have a default value of 0. When —server.threads is set to 0 on startup, the suitable number ofthreads will be determined by ArangoDB by asking the OS for the number of availableCPUs and using that as a baseline. If the number of CPUs is lower than 4, ArangoDBwill still start 4 dispatcher threads. When —scheduler.threads is set to 0,then ArangoDB will automatically determine the number of scheduler threads to start.This will normally create 2 scheduler threads.

If the exact number of threads needs to be set by the admin, then it is still possibleto set —server.threads and —scheduler.threads to non-zero values. ArangoDB willuse these values and start that many threads (note that some threads may be createdlazily so they may not be present directly after startup).

The number of V8 JavaScript contexts to be created (—javascript.v8-contexts) now has a default value of 0 too, meaning that ArangoDB will create as many V8 contextsas there will be dispatcher threads (controlled by the —server.threads option).Setting this option to a non-zero value will create exactly as many V8 contexts asspecified.

Setting these options explicitly to non-zero values may be beneficial in environmentsthat have few resources (processing time, maximum thread count, available memory).

Authentication

The default value for —server.authentication is now true in the configurationfiles shipped with ArangoDB. This means the server will be started with authentication enabled by default, requiring all client connections to provide authentication data when connecting to ArangoDB APIs. Previous ArangoDB versions used the setting—server.disable-authentication true, effectively disabling authentication by default.

The default value for —server.authentication-system-only is now true in ArangoDB.That means that Foxx applications running in ArangoDB will be public accessible (atleast they will not use ArangoDB’s builtin authentication mechanism). Only requests toArangoDB APIs at URL path prefixes /_api/ and /_admin will require authentication.To change that, and use the builtin authentication mechanism for Foxx applications too,set —server.authentication-system-only to false, and make sure to have the option—server.authentication set to true as well.

Though enabling the authentication is recommended for production setups, it may beoverkill in a development environment. To turn off authentication, the option —server.authentication can be set to false in ArangoDB’s configuration file oron the command-line.

Web Admin Interface

The JavaScript shell has been removed from ArangoDB’s web interface. The functionalitythe shell provided is still fully available in the ArangoShell (arangosh) binary shippedwith ArangoDB.

ArangoShell and client tools

The ArangoShell (arangosh) and the other client tools bundled with ArangoDB can onlyconnect to an ArangoDB server of version 3.0 or higher. They will not connect to anArangoDB 2.8. This is because the server HTTP APIs have changed between 2.8 and 3.0,and all client tools uses these APIs.

In order to connect to earlier versions of ArangoDB with the client tools, an olderversion of the client tools needs to be kept installed.

The preferred name for the template string generator function aqlQuery is nowaql and is automatically available in arangosh. Elsewhere, it can be loadedlike const aql = require('@arangodb').aql.

Command-line options added

All client tools in 3.0 provide an option —server.max-packet-size for controllingthe maximum size of HTTP packets to be handled by the client tools. The default valueis 128 MB, as in previous versions of ArangoDB. In contrast to previous versions inwhich the value was hard-coded, the option is now configurable. It can be increased tomake the client tools handle very large HTTP result messages sent by the server.

Command-line options changed

For all client tools, the option —server.disable-authentication was renamed to—server.authentication. Note that the meaning of the option —server.authentication is the opposite of the previous —server.disable-authentication.

The option —server.ssl-protocol was renamed to —ssl.protocol. The meaning of the option is unchanged.

The command-line option —quiet was removed from all client tools except arangosh because it had no effect in them.

Arangobench

In order to make its purpose more apparent the former arangob client tool has been renamed to arangobench in 3.0.

Miscellaneous changes

The checksum calculation algorithm for the collection.checksum() method and itscorresponding REST API GET /_api/collection/<collection</checksum has changed in 3.0. Checksums calculated in 3.0 will differ from checksums calculated with 2.8 or before.

The ArangoDB server in 3.0 does not read a file ENDPOINTS containing a list of additional endpoints on startup. In 2.8 this file was automatically read if presentin the database directory.

The names of the sub-threads started by ArangoDB have changed in 3.0. This is relevanton Linux only, where threads can be named and thread names may be visible to systemtools such as top or monitoring solutions.