Features and Improvements in ArangoDB 2.1

The following list shows in detail which features have been added or improved inArangoDB 2.1. ArangoDB 2.1 also contains several bugfixes that are not listedhere.

New Edges Index

The edges index (used to store connections between nodes in a graph) internallyuses a new data structure. This data structure improves the performance whenpopulating the edge index (i.e. when loading an edge collection). For largegraphs loading can be 20 times faster than with ArangoDB 2.0.

Additionally, the new index fixes performance problems that occurred when manyduplicate _from or _to values were contained in the index. Furthermore, thenew index supports faster removal of edges.

Finally, when loading an existing collection and building the edges index forthe collection, less memory re-allocations will be performed.

Overall, this should considerably speed up loading edge collections.

The new index type replaces the old edges index type automatically, without anychanges being required by the end user.

The API of the new index is compatible with the API of the old index. Still itis possible that the new index returns edges in a different order than the oldindex. This is still considered to be compatible because the old index had neverguaranteed any result order either.

AQL Improvements

AQL offers functionality to work with dates. Dates are no data types of their ownin AQL (neither they are in JSON, which is often used as a format to ship datainto and out of ArangoDB). Instead, dates in AQL are internally represented byeither numbers (timestamps) or strings. The date functions in AQL providemechanisms to convert from a numeric timestamp to a string representation andvice versa.

There are two date functions in AQL to create dates for further use:

  • DATE_TIMESTAMP(date) Creates a UTC timestamp value from date

  • DATE_TIMESTAMP(year, month, day, hour, minute, second, millisecond):Same as before, but allows specifying the individual date components separately.All parameters after day are optional.

  • DATE_ISO8601(date): Returns an ISO8601 datetime string from date.The datetime string will always use UTC time, indicated by the Z at its end.

  • DATE_ISO8601(year, month, day, hour, minute, second, millisecond):same as before, but allows specifying the individual date components separately.All parameters after day are optional.

These two above date functions accept the following input values:

  • numeric timestamps, indicating the number of milliseconds elapsed since the UNIXepoch (i.e. January 1st 1970 00:00:00 UTC).An example timestamp value is 1399472349522, which translates to 2014-05-07T14:19:09.522Z.

  • datetime strings in formats YYYY-MM-DDTHH:MM:SS.MMM, YYYY-MM-DD HH:MM:SS.MMM, or YYYY-MM-DD. Milliseconds are always optional.

A timezone difference may optionally be added at the end of the string, with thehours and minutes that need to be added or subtracted to the datetime value.For example, 2014-05-07T14:19:09+01:00 can be used to specify a one hour offset,and 2014-05-07T14:19:09+07:30 can be specified for seven and half hours offset. Negative offsets are also possible. Alternatively to an offset, a Z can be usedto indicate UTC / Zulu time.

An example value is 2014-05-07T14:19:09.522Z meaning May 7th 2014, 14:19:09 and 522 milliseconds, UTC / Zulu time. Another example value without time component is 2014-05-07Z.

Please note that if no timezone offset is specified in a datestring, ArangoDB willassume UTC time automatically. This is done to ensure portability of queries acrossservers with different timezone settings, and because timestamps will always beUTC-based.

  • individual date components as separate function arguments, in the following order:

    • year
    • month
    • day
    • hour
    • minute
    • second
    • millisecondAll components following day are optional and can be omitted. Note that notimezone offsets can be specified when using separate date components, and UTC /Zulu time will be used.

The following calls to DATE_TIMESTAMP are equivalent and will all return 1399472349522:

  1. DATE_TIMESTAMP("2014-05-07T14:19:09.522")
  2. DATE_TIMESTAMP("2014-05-07T14:19:09.522Z")
  3. DATE_TIMESTAMP("2014-05-07 14:19:09.522")
  4. DATE_TIMESTAMP("2014-05-07 14:19:09.522Z")
  5. DATE_TIMESTAMP(2014, 5, 7, 14, 19, 9, 522)
  6. DATE_TIMESTAMP(1399472349522)

The same is true for calls to DATE_ISO8601 that also accepts variable input formats:

  1. DATE_ISO8601("2014-05-07T14:19:09.522Z")
  2. DATE_ISO8601("2014-05-07 14:19:09.522Z")
  3. DATE_ISO8601(2014, 5, 7, 14, 19, 9, 522)
  4. DATE_ISO8601(1399472349522)

The above functions are all equivalent and will return "2014-05-07T14:19:09.522Z".

The following date functions can be used with dates created by DATE_TIMESTAMP andDATE_ISO8601:

  • DATE_DAYOFWEEK(date): Returns the weekday number of date. The return values havethe following meanings:
    • 0: Sunday
    • 1: Monday
    • 2: Tuesday
    • 3: Wednesday
    • 4: Thursday
    • 5: Friday
    • 6: Saturday

  • DATE_YEAR(date): Returns the year part of date as a number.

  • DATE_MONTH(date): Returns the month part of date as a number.

  • DATE_DAY(date): Returns the day part of date as a number.

  • DATE_HOUR(date): Returns the hour part of date as a number.

  • DATE_MINUTE(date): Returns the minute part of date as a number.

  • DATE_SECOND(date): Returns the seconds part of date as a number.

  • DATE_MILLISECOND(date): Returns the milliseconds part of date as a number.The following other date functions are also available:

  • DATE_NOW(): Returns the current time as a timestamp.

Note that this function is evaluated on every invocation and may return different values when invoked multiple times in the same query.

The following other AQL functions have been added in ArangoDB 2.1:

  • FLATTEN: this function can turn an array of sub-arrays into a single flat array. Allarray elements in the original array will be expanded recursively up to a configurabledepth. The expanded values will be added to the single result array.

Example:

  1. FLATTEN([ 1, 2, [ 3, 4 ], 5, [ 6, 7 ], [ 8, [ 9, 10 ] ])

will expand the sub-arrays on the first level and produce:

  1. [ 1, 2, 3, 4, 5, 6, 7, 8, [ 9, 10 ] ]

To fully flatten the array, the maximum depth can be specified (e.g. with a value of 2):

  1. FLATTEN([ 1, 2, [ 3, 4 ], 5, [ 6, 7 ], [ 8, [ 9, 10 ] ], 2)

This will fully expand the sub-arrays and produce:

  1. [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ]

  • CURRENT_DATABASE: this function will return the name of the database the currentquery is executed in.

  • CURRENT_USER: this function returns the name of the current user that is executingthe query. If authorization is turned off or the query is executed outside of a request context, no user is present and the function will return null.

Cluster Dump and Restore

The dump and restore tools, arangodump and arangorestore, can now be used todump and restore collections in a cluster. Additionally, a collection dump froma standalone ArangoDB server can be imported into a cluster, and vice versa.

Web Interface Improvements

The web interface in version 2.1 has a more compact dashboard. It providescharts with time-series for incoming requests, HTTP transfer volume and someserver resource usage figures.

Additionally it provides trend indicators (e.g. 15 min averages) anddistribution charts (aka histogram) for some figures.

Foxx Improvements

To easily access a file inside the directory of a Foxx application from withinFoxx, Foxx’s applicationContext now provides the foxxFilename() function. Itcan be used to assemble the full filename of a file inside the application’sdirectory. The applicationContext can be accessed as global variable from anymodule within a Foxx application.

The filename can be used inside Foxx actions or setup / teardown scripts,e.g. to populate a Foxx application’s collection with data.

The require function now also prefers local modules when used from inside aFoxx application. This allows putting modules inside the Foxx applicationdirectory and requiring them easily. It also allows using application-specificversions of libraries that are bundled with ArangoDB (such as underscore.js).

Windows Installer

The Windows installer shipped with ArangoDB now supports installation ofArangoDB for the current user or all users, with the required privileges. Italso supports the installation of ArangoDB as a service.

Fixes for 32 bit systems

Several issues have been fixed that occurred only when using ArangoDB on a 32 bitsoperating system, specifically:

  • a crash in a third party component used to manage cluster data

  • a third party library that failed to initialize on 32 bit Windows, making arangodand arangosh crash immediately.

  • overflows of values used for nanosecond-precision timeouts: these overflowshave led to invalid values being passed to socket operations, making them failand re-try too often

Updated drivers

Several drivers for ArangoDB have been checked for compatibility with 2.1. Thecurrent list of drivers with compatibility notes can be found onlinehere.

C++11 usage

We have moved several files from C to C++, allowing more code reuse and reducingthe need for shipping data between the two. We have also decided to requireC++11 support for ArangoDB, which allows us to use some of the simplifications,features and guarantees that this standard has in stock.

That also means a compiler with C++11 support is required to build ArangoDB fromsource. For instance GNU CC of at least version 4.8.

Miscellaneous Improvements

  • Cancelable asynchronous jobs: several potentially long-running jobs can now becanceled via an explicit cancel operation. This allows stopping long-runningqueries, traversals or scripts without shutting down the complete ArangoDBprocess. Job cancelation is provided for asynchronously executed jobs as isdescribed in @ref HttpJobCancel.

  • Server-side periodic task management: an ArangoDB server now providesfunctionality to register and unregister periodic tasks. Tasks areuser-defined JavaScript actions that can be run periodically andautomatically, independent of any HTTP requests.

The following task management functions are provided:

  • require(“org/arangodb/tasks”).register(): registers a periodic task
  • require(“org/arangodb/tasks”).unregister(): unregisters and removes a periodic task
  • require(“org/arangodb/tasks”).get(): retrieves a specific tasks or all existing tasksAn example task (to be executed every 15 seconds) can be registered like this:
  1. var tasks = require("org/arangodb/tasks");
  2. tasks.register({
  3.   name: "this is an example task with parameters",
  4.   period: 15,
  5.   command: function (params) {
  6.     var greeting = params.greeting;
  7.     var data = JSON.stringify(params.data);
  8.     require('console').log('%s from parameter task: %s', greeting, data);
  9.   },
  10.   params: { greeting: "hi", data: "how are you?" }
  11. });

Please refer to the section @ref Tasks for more details.

  • The figures method of a collection now returns data about the collection’sindex memory consumption. The returned value indexes.size will contain thetotal amount of memory acquired by all indexes of the collection. This figurecan be used to assess the memory impact of indexes.

  • Capitalized HTTP response headers: from version 2.1, ArangoDB will returncapitalized HTTP headers by default, e.g. Content-Length instead ofcontent-length. Though the HTTP specification states that headers fieldname are case-insensitive, several older client tools rely on a specific casein HTTP response headers. This changes make ArangoDB a bit more compatiblewith those.

  • Simplified usage of db._createStatement(): to easily run an AQL query, themethod db._createStatement now allows passing the AQL query as a string.Previously it required the user to pass an object with a query attribute(which then contained the query string).

ArangoDB now supports both versions:

  1. db._createStatement(queryString);
  2. db._createStatement({ query: queryString });