Skip to content

Commit 822c172

Browse files
committed
DOCS-686 incorporate comments from antoine and add the troubleshoot page
1 parent 04975c2 commit 822c172

File tree

4 files changed

+382
-65
lines changed

4 files changed

+382
-65
lines changed

source/applications/map-reduce.txt

Lines changed: 57 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -4,37 +4,47 @@ Map-Reduce
44

55
.. default-domain:: mongodb
66

7-
Map-reduce operations can handle complex aggregation
8-
tasks. [#simple-aggregation-use-framework]_ To perform map-reduce operations,
9-
MongoDB provides the :dbcommand:`mapReduce` command and, in the
10-
:program:`mongo` shell, the wrapper :method:`db.collection.mapReduce()`
11-
method.
7+
Map-reduce operations can handle complex aggregation tasks. To perform
8+
map-reduce operations, MongoDB provides the :dbcommand:`mapReduce`
9+
command and, in the :program:`mongo` shell, the
10+
:method:`db.collection.mapReduce()` wrapper method.
1211

13-
This overview will cover:
12+
.. contents:: This overview will cover:
13+
:backlinks: none
14+
:local:
15+
:depth: 1
1416

15-
- :ref:`map-reduce-method`
17+
For many simple aggregation tasks, see the :doc:`aggregation framework
18+
</applications/aggregation>`.
1619

17-
- :ref:`map-reduce-examples`
18-
19-
- :ref:`map-reduce-incremental`
20-
21-
- :ref:`map-reduce-sharded-cluster`
20+
.. _map-reduce-examples:
2221

23-
- :ref:`map-reduce-additional-references`
22+
Map-Reduce Examples
23+
-------------------
2424

25-
.. _map-reduce-method:
25+
This section provides some map-reduce examples in the :program:`mongo`
26+
shell using the :method:`db.collection.mapReduce()` method:
2627

27-
mapReduce()
28-
-----------
28+
.. code-block:: javascript
2929

30-
.. include:: /reference/method/db.collection.mapReduce.txt
31-
:start-after: mongodb
32-
:end-before: mapReduce-syntax-end
30+
db.collection.mapReduce(
31+
<map>,
32+
<reduce>,
33+
{
34+
<out>,
35+
<query>,
36+
<sort>,
37+
<limit>,
38+
<finalize>,
39+
<scope>,
40+
<jsMode>,
41+
<verbose>
42+
}
43+
)
44+
45+
For more information on the parameters, see the
46+
:method:`db.collection.mapReduce()` reference page .
3347

34-
.. _map-reduce-examples:
35-
36-
Map-Reduce Examples
37-
-------------------
3848
.. include:: /includes/examples-map-reduce.rst
3949
:start-after: map-reduce-examples-begin
4050
:end-before: map-reduce-sum-price-wrapper-end
@@ -140,10 +150,10 @@ each day and can be simulated as follows:
140150

141151
var finalizeFunction = function (key, reducedValue) {
142152

143-
if (reducedValue.count > 0)
144-
reducedValue.avg_time = reducedValue.total_time / reducedValue.count;
153+
if (reducedValue.count > 0)
154+
reducedValue.avg_time = reducedValue.total_time / reducedValue.count;
145155

146-
return reducedValue;
156+
return reducedValue;
147157
};
148158

149159
#. Perform map-reduce on the ``session`` collection using the
@@ -154,26 +164,24 @@ each day and can be simulated as follows:
154164

155165
.. code-block:: javascript
156166

157-
db.runCommand(
158-
{
159-
mapreduce: "sessions",
160-
map: mapFunction,
161-
reduce:reduceFunction,
162-
out: { reduce: "session_stat" },
163-
finalize: finalizeFunction
164-
}
165-
);
167+
db.sessions.mapReduce( mapFunction,
168+
reduceFunction,
169+
{
170+
out: { reduce: "session_stat" },
171+
finalize: finalizeFunction
172+
}
173+
)
166174

167175
**Subsequent Incremental Map-Reduce**
168176

169177
Assume the next day, the ``sessions`` collection grows by the following documents:
170178

171179
.. code-block:: javascript
172180

173-
db.session.save( { userid: "a", ts: ISODate('2011-11-05 14:17:00'), length: 100 } );
174-
db.session.save( { userid: "b", ts: ISODate('2011-11-05 14:23:00'), length: 115 } );
175-
db.session.save( { userid: "c", ts: ISODate('2011-11-05 15:02:00'), length: 125 } );
176-
db.session.save( { userid: "d", ts: ISODate('2011-11-05 16:45:00'), length: 55 } );
181+
db.sessions.save( { userid: "a", ts: ISODate('2011-11-05 14:17:00'), length: 100 } );
182+
db.sessions.save( { userid: "b", ts: ISODate('2011-11-05 14:23:00'), length: 115 } );
183+
db.sessions.save( { userid: "c", ts: ISODate('2011-11-05 15:02:00'), length: 125 } );
184+
db.sessions.save( { userid: "d", ts: ISODate('2011-11-05 16:45:00'), length: 55 } );
177185

178186
5. At the end of the day, perform incremental map-reduce on the
179187
``sessions`` collection but use the ``query`` field to select only the
@@ -183,15 +191,14 @@ Assume the next day, the ``sessions`` collection grows by the following document
183191

184192
.. code-block:: javascript
185193

186-
db.runCommand( {
187-
mapreduce: "sessions",
188-
map: mapFunction,
189-
reduce:reduceFunction,
190-
query: { ts: { $gt: ISODate('2011-11-05 00:00:00') } },
191-
out: { reduce: "session_stat" },
192-
finalize:finalizeFunction
193-
}
194-
);
194+
db.sessions.mapReduce( mapFunction,
195+
reduceFunction,
196+
{
197+
query: { ts: { $gt: ISODate('2011-11-05 00:00:00') } },
198+
out: { reduce: "session_stat" },
199+
finalize: finalizeFunction
200+
}
201+
);
195202

196203
.. _map-reduce-temporay-collection:
197204

@@ -277,12 +284,9 @@ Additional References
277284

278285
.. seealso::
279286

287+
- :doc:`/tutorial/troubleshoot-map-reduce`
288+
280289
- :wiki:`Map-Reduce Concurrency
281290
<How+does+concurrency+work#Howdoesconcurrencywork-MapReduce>`
282291

283292
- `MapReduce, Geospatial Indexes, and Other Cool Features <http://www.slideshare.net/mongosf/mapreduce-geospatial-indexing-and-other-cool-features-kristina-chodorow>`_ - Kristina Chodorow at MongoSF (April 2010)
284-
285-
- :wiki:`Troubleshooting MapReduce`
286-
287-
.. [#simple-aggregation-use-framework] For many simple aggregation tasks, see the
288-
:doc:`aggregation framework </applications/aggregation>`.

source/includes/examples-map-reduce.rst

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,8 @@ Perform map-reduce operation on the ``orders`` collection to group by
2929
the ``cust_id``, and for each ``cust_id``, calculate the sum of the
3030
``price`` for each ``cust_id``:
3131

32+
.. map-reduce-map-function-begin
33+
3234
#. Define the ``<map>`` function to process each document in the
3335
map-reduce process:
3436

@@ -44,6 +46,8 @@ the ``cust_id``, and for each ``cust_id``, calculate the sum of the
4446
emit(this.cust_id, this.price);
4547
};
4648
49+
.. map-reduce-map-function-end
50+
4751
#. Define the corresponding ``<reduce>`` function with two arguments
4852
``keyCustId`` and ``valuesPrices``:
4953

source/includes/parameters-map-reduce.rst

Lines changed: 38 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -31,8 +31,8 @@
3131
- The ``emit(key,value)`` function associates the ``key``
3232
with a ``value``.
3333

34-
- Each "emit" is limited to half the MongoDB :limit:`BSON
35-
document size`.
34+
- Each "emit" is limited to half the MongoDB :ref:`maximum
35+
BSON document size <limit-bson-document-size>`.
3636

3737
- There is no limit to the number of times you may call the
3838
``emit`` function per document.
@@ -68,11 +68,32 @@
6868
- The ``<reduce>`` function should *not* affect the outside
6969
system.
7070

71+
- The ``<reduce>`` function is *idempotent*; i.e. the
72+
following behavior holds:
73+
74+
.. code-block:: javascript
75+
76+
reduce( key, [ reduce(key, valuesArray) ] ) == reduce ( key, valuesArray )
77+
78+
Additionally, the order of the elements in the
79+
``valuesArray`` does not affect the result:
80+
81+
.. code-block:: javascript
82+
83+
reduce ( key, [ A, B ] ) == reduce ( key, [ B, A ] )
84+
7185
- Because it is possible to invoke the ``<reduce>`` function
7286
more than once for the same key, the *type* of the return
7387
object must be **identical** to the type of the ``value``
74-
emitted by the ``<map>`` function.
75-
88+
emitted by the ``<map>`` function to ensure that:
89+
90+
.. code-block:: javascript
91+
92+
reduce(key, [ C, reduce(key, [ A, B ]) ] ) == reduce (key, [ C, A, B ] )
93+
94+
- The ``<reduce>`` function is **not** invoked for a key
95+
that has only a single value.
96+
7697
- The ``<reduce>`` function can access the variables defined
7798
in the ``<scope>`` parameter if the ``<scope>`` parameter
7899
is defined for the map-reduce operation.
@@ -151,12 +172,15 @@
151172

152173
.. versionadded:: 2.1
153174

154-
Optional. Specify output operation as non-atomic. If
155-
``true``, the post processing step will not execute
156-
inside of a database lock so that partial results will
157-
be visible during processing . ``nonAtomic`` is valid
158-
*only* for ``merge`` and ``reduce`` output operations
159-
where post-processing may be a long-running operation.
175+
Optional. Specify output operation as non-atomic and is
176+
valid *only* for ``merge`` and ``reduce`` output modes.
177+
Post-processing for ``merge`` and ``reduce`` output
178+
modes may take a long time (e.g. minutes). During this
179+
time, the entire database is locked for both reads and
180+
writes. If ``nonAtomic`` is ``true``, the post
181+
processing step will prevent the locking of the
182+
database; however, partial results will be visible as
183+
they are processed.
160184

161185
- **Output inline**. Perform the map-reduce operation in memory
162186
and return the result. This option is the only available
@@ -246,8 +270,10 @@
246270
- Can only work for result sets with less than 500,000 distinct
247271
``key`` arguments to the mapper's ``emit()`` function.
248272

249-
The ``<jsMode>`` defaults to true.
273+
The ``<jsMode>`` defaults to false.
250274

251275
:param Boolean verbose:
252276

253-
Optional. Provides statistics on job execution times.
277+
Optional. Specifies whether to include the ``timing``
278+
information in the result information. The ``<verbose>``
279+
defaults to ``true`` to include the ``timing`` information.

0 commit comments

Comments
 (0)