-
Notifications
You must be signed in to change notification settings - Fork 246
DRIVERS-2993 specify behavior of w:0 with MongoClient.bulkWrite
#1670
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 7 commits
06697b5
e6f710b
6875551
0a9c3ab
c5c7407
1ecd88d
977fde1
13dbbbe
02696e8
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -397,7 +397,8 @@ InsertOne: { | |
|
|
||
| Construct as list of write models (referred to as `models`) with the one `model`. | ||
|
|
||
| Call `MongoClient.bulkWrite` with `models` and `BulkWriteOptions.writeConcern` set to an unacknowledged write concern. | ||
| Call `MongoClient.bulkWrite` with `models`. Pass `BulkWriteOptions` with `ordered` set to `false` and `writeConcern` set | ||
| to an unacknowledged write concern. | ||
|
|
||
| Expect a client-side error due the size. | ||
|
|
||
|
|
@@ -415,7 +416,8 @@ ReplaceOne: { | |
|
|
||
| Construct as list of write models (referred to as `models`) with the one `model`. | ||
|
|
||
| Call `MongoClient.bulkWrite` with `models` and `BulkWriteOptions.writeConcern` set to an unacknowledged write concern. | ||
| Call `MongoClient.bulkWrite` with `models`. Pass `BulkWriteOptions` with `ordered` set to `false` and `writeConcern` set | ||
| to an unacknowledged write concern. | ||
|
|
||
| Expect a client-side error due the size. | ||
|
|
||
|
|
@@ -693,3 +695,43 @@ maxTimeMS value of 2000ms for the `explain`. | |
|
|
||
| Obtain the command started event for the explain. Confirm that the top-level explain command should has a `maxTimeMS` | ||
| value of `2000`. | ||
|
|
||
| ### 15. `MongoClient.bulkWrite` with unacknowledged write concern uses `w:0` for all batches | ||
|
|
||
| This test must only be run on 8.0+ servers. This test must be skipped on Atlas Serverless. | ||
|
|
||
| If testing with a sharded cluster, only connect to one mongos. This is intended to ensure the `countDocuments` operation | ||
| uses the same connection as the `bulkWrite` to get the correct connection count. (See | ||
| [DRIVERS-2921](https://jira.mongodb.org/browse/DRIVERS-2921)). | ||
|
|
||
| Construct a `MongoClient` (referred to as `client`) with | ||
| [command monitoring](../../command-logging-and-monitoring/command-logging-and-monitoring.md) enabled to observe | ||
| CommandStartedEvents. Perform a `hello` command using `client` and record the `maxWriteBatchSize` value contained in the | ||
| response. | ||
|
|
||
| Construct a `MongoCollection` (referred to as `coll`) for the collection "db.coll". Drop `coll`. | ||
|
|
||
| Use the `create` command to create "db.coll" to workaround [SERVER-95537](https://jira.mongodb.org/browse/SERVER-95537). | ||
|
|
||
| Construct the following write model (referred to as `model`): | ||
|
|
||
| ```javascript | ||
| InsertOne: { | ||
| "namespace": "db.coll", | ||
| "document": { "a": "b" } | ||
| } | ||
| ``` | ||
|
|
||
| Construct a list of write models (referred to as `models`) with `model` repeated `maxWriteBatchSize + 1` times. | ||
|
||
|
|
||
| Call `client.bulkWrite` with `models`. Pass `BulkWriteOptions` with `ordered` set to `false` and `writeConcern` set to | ||
| an unacknowledged write concern. Assert no error occurred. Assert the result indicates the write was unacknowledged. | ||
|
|
||
| Assert that two CommandStartedEvents (referred to as `firstEvent` and `secondEvent`) were observed for the `bulkWrite` | ||
| command. Assert that the length of `firstEvent.command.ops` is `maxWriteBatchSize`. Assert that the length of | ||
| `secondEvent.command.ops` is 1. If the driver exposes `operationId`s in its CommandStartedEvents, assert that | ||
| `firstEvent.operationId` is equal to `secondEvent.operationId`. Assert both commands include `writeConcern: {w: 0}`. | ||
|
|
||
| To force completion of the `w:0` writes, execute `coll.countDocuments` and expect the returned count is | ||
| `maxWriteBatchSize + 1`. This is intended to avoid incomplete writes interfering with other tests that may use this | ||
| collection. | ||
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What should drivers that already released this api do? Should we continue supporting it until the next major version? Drop it in a minor version? We usually try to never do the latter but I'd be willing to make an expection here since unack'd writes are so niche and the fix is simple, just add ordered=False.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this only currently applies to Python and C (Rust does not implement unacknowledged writes). I was not planning to wait for a major release to fix in C.
Arguably, the previous behavior of allowing ordered+
w:0is a bug, since the behavior does not match the description:There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just to clarify, Python does not have that bug. We send w:1 so we can stop if a batch fails.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ShaneHarvey: what is the bug you're referring to? I'm a bit confused how PyMongo sending
w:1relates to the user specifyingw:0withordered:true. Does that mean PyMongo is already disregarding the user's original write concern in order to catch failed batches?Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes that is what drivers are supposed to do. With ordered:true + w:0, all but the last batch use w:1 and only the final batch uses w:0. I'm not sure if this is specified anywhere but that's how pymongo's collection.bulk_write works.
That said, I'm fine with dropping support for this in our next release (for client.bulk_write only).