db.collection.mapReduce()

  • db.collection.mapReduce(map, reduce, {, , , , , , , })

mongo Shell Method

This page documents the mongo shell method, and doesnot refer to the MongoDB Node.js driver (or any other driver)method. For corresponding MongoDB driver API, refer to your specificMongoDB driver documentation instead.

Note

Starting in version 4.2, MongoDB deprecates:

  • The map-reduce option to create a new sharded collection as wellas the use of the sharded option formap-reduce. To output to a sharded collection, create the shardedcollection first. MongoDB 4.2 also deprecates the replacement ofan existing sharded collection.
  • The explicit specification of nonAtomic: false option.

The db.collection.mapReduce() method provides a wrapperaround the mapReduce command.

Note

Views do not support map-reduce operations.

db.collection.mapReduce() has the following syntax:

  1. db.collection.mapReduce(
  2.                          <map>,
  3.                          <reduce>,
  4.                          {
  5.                            out: <collection>,
  6.                            query: <document>,
  7.                            sort: <document>,
  8.                            limit: <number>,
  9.                            finalize: <function>,
  10.                            scope: <document>,
  11.                            jsMode: <boolean>,
  12.                            verbose: <boolean>,
  13.                            bypassDocumentValidation: <boolean>
  14.                          }
  15.                        )

db.collection.mapReduce() takes the following parameters:

ParameterTypeDescriptionmapfunctionA JavaScript function that associates or “maps” a value with akey and emits the key and value pair.

See Requirements for the map Function for more information.reducefunctionA JavaScript function that “reduces” to a single object all thevalues associated with a particular key.

See Requirements for the reduce Function for more information.optionsdocumentA document that specifies additional parameters todb.collection.mapReduce().bypassDocumentValidationbooleanOptional. Enables mapReduce to bypass document validationduring the operation. This lets you insert documents that do notmeet the validation requirements.

New in version 3.2.

The following table describes additional arguments thatdb.collection.mapReduce() can accept.

FieldTypeDescriptionoutstring or documentSpecifies the location of the result of the map-reduce operation.You can output to a collection, output to a collection with anaction, or output inline. You may output to a collection whenperforming map-reduce operations on the primary members of the set;on secondary members you may only use the inline output.

See out Options for more information.querydocumentSpecifies the selection criteria using query operators for determining the documents input to themap function.sortdocumentSorts the input documents. This option is useful foroptimization. For example, specify the sort key to be the same asthe emit key so that there are fewer reduce operations. The sort keymust be in an existing index for this collection.limitnumberSpecifies a maximum number of documents for the input into themap function.finalizefunctionOptional. Follows the reduce method and modifies the output.

See Requirements for the finalize Function for more information.scopedocumentSpecifies global variables that are accessible in the map,reduce and finalize functions.jsModebooleanSpecifies whether to convert intermediate data into BSONformat between the execution of the map and reducefunctions.

Defaults to false.

If false:

  • Internally, MongoDB converts the JavaScript objects emittedby the mapfunction to BSON objects. These BSONobjects are then converted back to JavaScript objects whencalling the reduce function.

  • The map-reduce operation places the intermediate BSON objectsin temporary, on-disk storage. This allows the map-reduceoperation to execute over arbitrarily large data sets.If true:

  • Internally, the JavaScript objects emitted during mapfunction remain as JavaScript objects. There is no need toconvert the objects for the reduce function, whichcan result in faster execution.

  • You can only use jsMode for result sets with fewer than500,000 distinct key arguments to the mapper’s emit()function.verbosebooleanSpecifies whether to include the timing information in theresult information. Set verbose to true to includethe timing information.

Defaults to false.collationdocumentOptional.

Specifies the collation to use for the operation.

Collation allows users to specifylanguage-specific rules for string comparison, such as rules forlettercase and accent marks.

The collation option has the following syntax:

  1. collation: {
  2.    locale: <string>,
  3.    caseLevel: <boolean>,
  4.    caseFirst: <string>,
  5.    strength: <int>,
  6.    numericOrdering: <boolean>,
  7.    alternate: <string>,
  8.    maxVariable: <string>,
  9.    backwards: <boolean>
  10. }

When specifying collation, the locale field is mandatory; allother collation fields are optional. For descriptions of the fields,see Collation Document.

If the collation is unspecified but the collection has adefault collation (see db.createCollection()), theoperation uses the collation specified for the collection.

If no collation is specified for the collection or for theoperations, MongoDB uses the simple binary comparison used in priorversions for string comparisons.

You cannot specify multiple collations for an operation. Forexample, you cannot specify different collations per field, or ifperforming a find with a sort, you cannot use one collation for thefind and another for the sort.

New in version 3.4.

Note

map-reduce operations and $whereoperator expressions cannot access certain global functions orproperties, such as db, that are available in themongo shell.

The following JavaScript functions and properties are available tomap-reduce operations and $whereoperator expressions:

Available PropertiesAvailable Functions

args

MaxKey

MinKey

assert()

BinData()

DBPointer()

DBRef()

doassert()

emit()

gc()

HexData()

hex_md5()

isNumber()

isObject()

ISODate()

isString()

Map()

MD5()

NumberInt()

NumberLong()

ObjectId()

print()

printjson()

printjsononeline()

sleep()

Timestamp()

tojson()

tojsononeline()

tojsonObject()

UUID()

version()

Requirements for the map Function

The map function is responsible for transforming each input document intozero or more documents. It can access the variables defined in the scopeparameter, and has the following prototype:

  1. function() {
  2.    ...
  3.    emit(key, value);
  4. }

The map function has the following requirements:

  • In the map function, reference the current document as thiswithin the function.
  • The map function should not access the database for any reason.
  • The map function should be pure, or have no impact outside ofthe function (i.e. side effects.)
  • A single emit can only hold half of MongoDB’s maximum BSONdocument size.
  • The map function may optionally call emit(key,value) any number oftimes to create an output document associating key with value.
  • Starting in version 4.2.1, MongoDB deprecates the use of JavaScriptwith scope (i.e. BSON type 15) forthe map function. To scope variables, use the scope parameterinstead.

The following map function will call emit(key,value) either0 or 1 times depending on the value of the input document’sstatus field:

  1. function() {
  2.     if (this.status == 'A')
  3.         emit(this.cust_id, 1);
  4. }

The following map function may call emit(key,value)multiple times depending on the number of elements in the inputdocument’s items field:

  1. function() {
  2.     this.items.forEach(function(item){ emit(item.sku, 1); });
  3. }

Requirements for the reduce Function

The reduce function has the following prototype:

  1. function(key, values) {
  2.    ...
  3.    return result;
  4. }

The reduce function exhibits the following behaviors:

  • The reduce function should not access the database,even to perform read operations.
  • The reduce function should not affect the outsidesystem.
  • MongoDB will not call the reduce function for a keythat has only a single value. The values argument is an arraywhose elements are the value objects that are “mapped” to thekey.
  • MongoDB can invoke the reduce function more than once for thesame key. In this case, the previous output from the reducefunction for that key will become one of the input values to the nextreduce function invocation for that key.
  • The reduce function can access the variables definedin the scope parameter.
  • The inputs to reduce must not be larger than half of MongoDB’smaximum BSON document size. Thisrequirement may be violated when large documents are returned and thenjoined together in subsequent reduce steps.
  • Starting in version 4.2.1, MongoDB deprecates the use of JavaScriptwith scope (i.e. BSON type 15) forthe reduce function. To scope variables, use the scopeparameter instead.

Because it is possible to invoke the reduce functionmore than once for the same key, the followingproperties need to be true:

  • the type of the return object must be identicalto the type of the value emitted by the mapfunction.

  • the reduce function must be associative. The following statement must betrue:

  1. reduce(key, [ C, reduce(key, [ A, B ]) ] ) == reduce( key, [ C, A, B ] )
  • the reduce function must be idempotent. Ensurethat the following statement is true:
  1. reduce( key, [ reduce(key, valuesArray) ] ) == reduce( key, valuesArray )
  • the reduce function should be commutative: that is, the order of theelements in the valuesArray should not affect the output of thereduce function, so that the following statement is true:
  1. reduce( key, [ A, B ] ) == reduce( key, [ B, A ] )

out Options

You can specify the following options for the out parameter:

Output to a Collection

This option outputs to a new collection, and is not available on secondarymembers of replica sets.

  1. out: <collectionName>

Output to a Collection with an Action

Note

Starting in version 4.2, MongoDB deprecates:

  • The map-reduce option to create a new sharded collection as wellas the use of the sharded option formap-reduce. To output to a sharded collection, create the shardedcollection first. MongoDB 4.2 also deprecates the replacement ofan existing sharded collection.
  • The explicit specification of nonAtomic: false option.

This option is only available when passing a collection thatalready exists to out. It is not availableon secondary members of replica sets.

  1. out: { <action>: <collectionName>
  2.         [, db: <dbName>]
  3.         [, sharded: <boolean> ]
  4.         [, nonAtomic: <boolean> ] }

When you output to a collection with an action, the out has thefollowing parameters:

  • <action>: Specify one of the following actions:

    • replace

Replace the contents of the <collectionName> if thecollection with the <collectionName> exists.

  • merge

Merge the new result with the existing result if theoutput collection already exists. If an existing documenthas the same key as the new result, overwrite thatexisting document.

  • reduce

Merge the new result with the existing result if theoutput collection already exists. If an existing documenthas the same key as the new result, apply the reducefunction to both the new and the existing documents andoverwrite the existing document with the result.

  • db:

Optional. The name of the database that you want the map-reduceoperation to write its output. By default this will be the samedatabase as the input collection.

  • sharded:

Note

Starting in version 4.2, the use of the sharded option isdeprecated.

Optional. If true and you have enabled sharding on outputdatabase, the map-reduce operation will shard the output collectionusing the _id field as the shard key.

If true and collectionName is an existing unsharded collection,map-reduce fails.

  • nonAtomic:

Note

Starting in MongoDB 4.2, explicitly setting nonAtomic to false isdeprecated.

Optional. Specify output operation as non-atomic. This applies onlyto the merge and reduce output modes, which may take minutes toexecute.

By default nonAtomic is false, and the map-reduceoperation locks the database during post-processing.

If nonAtomic is true, the post-processing step preventsMongoDB from locking the database: during this time, other clientswill be able to read intermediate states of the output collection.

Output Inline

Perform the map-reduce operation in memory and return the result. Thisoption is the only available option for out on secondary members ofreplica sets.

  1. out: { inline: 1 }

The result must fit within the maximum size of a BSON document.

Requirements for the finalize Function

The finalize function has the following prototype:

  1. function(key, reducedValue) {
  2.    ...
  3.    return modifiedObject;
  4. }

The finalize function receives as its arguments a keyvalue and the reducedValue from the reduce function. Beaware that:

  • The finalize function should not access the database forany reason.
  • The finalize function should be pure, or have no impactoutside of the function (i.e. side effects.)
  • The finalize function can access the variables defined inthe scope parameter.
  • Starting in version 4.2.1, MongoDB deprecates the use of JavaScriptwith scope (i.e. BSON type 15) forthe finalize function. To scope variables, use the scopeparameter instead.

Map-Reduce Examples

Consider the following map-reduce operations on a collectionorders that contains documents of the following prototype:

  1. {
  2.      _id: ObjectId("50a8240b927d5d8b5891743c"),
  3.      cust_id: "abc123",
  4.      ord_date: new Date("Oct 04, 2012"),
  5.      status: 'A',
  6.      price: 25,
  7.      items: [ { sku: "mmm", qty: 5, price: 2.5 },
  8.               { sku: "nnn", qty: 5, price: 2.5 } ]
  9. }

Return the Total Price Per Customer

Perform the map-reduce operation on the orders collection to groupby the cust_id, and calculate the sum of the price for eachcust_id:

  • Define the map function to process each input document:

    • In the function, this refers to the document that themap-reduce operation is processing.
    • The function maps the price to the cust_id for eachdocument and emits the cust_id and price pair.
  1. var mapFunction1 = function() {
  2.                        emit(this.cust_id, this.price);
  3.                    };

  • Define the corresponding reduce function with two argumentskeyCustId and valuesPrices:

    • The valuesPrices is an array whose elements are the pricevalues emitted by the map function and grouped by keyCustId.
    • The function reduces the valuesPrice array to thesum of its elements.
  1. var reduceFunction1 = function(keyCustId, valuesPrices) {
  2.                           return Array.sum(valuesPrices);
  3.                       };
  • Perform the map-reduce on all documents in the orders collectionusing the mapFunction1 map function and the reduceFunction1reduce function.
  1. db.orders.mapReduce(
  2.                      mapFunction1,
  3.                      reduceFunction1,
  4.                      { out: "map_reduce_example" }
  5.                    )

This operation outputs the results to a collection namedmap_reduce_example. If the map_reduce_example collectionalready exists, the operation will replace the contents with theresults of this map-reduce operation:

Calculate Order and Total Quantity with Average Quantity Per Item

In this example, you will perform a map-reduce operation on theorders collection for all documents that have an ord_datevalue greater than 01/01/2012. The operation groups by theitem.sku field, and calculates the number oforders and the total quantity ordered for each sku. The operation concludes bycalculating the average quantity per order for each sku value:

  • Define the map function to process each input document:

    • In the function, this refers to the document that themap-reduce operation is processing.
    • For each item, the function associates the sku with a newobject value that contains the count of 1 and theitem qty for the order and emits the sku and value pair.
  1. var mapFunction2 = function() {
  2.                        for (var idx = 0; idx < this.items.length; idx++) {
  3.                            var key = this.items[idx].sku;
  4.                            var value = {
  5.                                          count: 1,
  6.                                          qty: this.items[idx].qty
  7.                                        };
  8.                            emit(key, value);
  9.                        }
  10.                     };

  • Define the corresponding reduce function with two argumentskeySKU and countObjVals:

    • countObjVals is an array whose elements are the objectsmapped to the grouped keySKU values passed by mapfunction to the reducer function.
    • The function reduces the countObjVals array to a singleobject reducedValue that contains the count and theqty fields.
    • In reducedVal, the count field contains the sum of thecount fields from the individual array elements, and theqty field contains the sum of the qty fields from theindividual array elements.
  1. var reduceFunction2 = function(keySKU, countObjVals) {
  2.                      reducedVal = { count: 0, qty: 0 };
  3.  
  4.                      for (var idx = 0; idx < countObjVals.length; idx++) {
  5.                          reducedVal.count += countObjVals[idx].count;
  6.                          reducedVal.qty += countObjVals[idx].qty;
  7.                      }
  8.  
  9.                      return reducedVal;
  10.                   };
  • Define a finalize function with two arguments key andreducedVal. The function modifies the reducedVal objectto add a computed field named avg and returns the modifiedobject:
  1. var finalizeFunction2 = function (key, reducedVal) {
  2.  
  3.                        reducedVal.avg = reducedVal.qty/reducedVal.count;
  4.  
  5.                        return reducedVal;
  6.  
  7.                     };
  • Perform the map-reduce operation on the orders collection usingthe mapFunction2, reduceFunction2, andfinalizeFunction2 functions.
  1. db.orders.mapReduce( mapFunction2,
  2.                      reduceFunction2,
  3.                      {
  4.                        out: { merge: "map_reduce_example" },
  5.                        query: { ord_date:
  6.                                   { $gt: new Date('01/01/2012') }
  7.                               },
  8.                        finalize: finalizeFunction2
  9.                      }
  10.                    )

This operation uses the query field to select only thosedocuments with ord_date greater than newDate(01/01/2012). Then it output the results to a collectionmap_reduce_example. If the map_reduce_example collectionalready exists, the operation will merge the existing contents withthe results of this map-reduce operation.

Output

The output of the db.collection.mapReduce() method isidentical to that of the mapReduce command. See theOutput section of the mapReducecommand for information on the db.collection.mapReduce()output.

Restrictions

MongoDB drivers automatically set afterClusterTime for operations associated with causallyconsistent sessions. Starting in MongoDB 4.2, thedb.collection.mapReduce() no longer supportafterClusterTime. As such,db.collection.mapReduce() cannot be associatd withcausally consistent sessions.

Additional Information