MongoDB: Transaction support

In database world, a transaction represents a single unit of work and represents the change in a database.

 

A database transaction must satisfy following properties.

a.   Atomicity: Transaction must either be complete in its entirety or have no effect whatsoever

b.   Consistency: Transaction must conform to existing constraints in the database

c.    Isolation: Transaction must not affect other transactions

d.   Durability: Changes to the database get written to persistent storage.

 

MongoDB maintains atomicity at document level by default. That means, when you are writing a document to mongoDB, it writes it completely else nothing will be written.

 

But if you are working with multiple documents, and multiple collections, you need to use transactions to achieve this atomicity.

 

For example, let’s model an author and book relationship.

 

Define authors collection

db.authors.insertMany([
	{
		"firstName": "Rama Krishna",
		"lastName" : "Gurram"
	},
	{
		"firstName" : "Sailaja",
		"lastName" : "Ptr"
	}
])

> db.authors.insertMany([
... {
... "firstName": "Rama Krishna",
... "lastName" : "Gurram"
... },
... {
... "firstName" : "Sailaja",
... "lastName" : "Ptr"
... }
... ])
{
	"acknowledged" : true,
	"insertedIds" : [
		ObjectId("60cae3c0fc455868b4f32103"),
		ObjectId("60cae3c0fc455868b4f32104")
	]
}
> 
>
>
> db.authors.find().pretty()
{
	"_id" : ObjectId("60cae3c0fc455868b4f32103"),
	"firstName" : "Rama Krishna",
	"lastName" : "Gurram"
}
{
	"_id" : ObjectId("60cae3c0fc455868b4f32104"),
	"firstName" : "Sailaja",
	"lastName" : "Ptr"
}

 

Let’s define books collection

db.books.insertMany([
	{
		"bookName": "Programming Puzzles",
		"price": "450",
		"noOfPage": 345,
		"authorId" : ObjectId("60cae3c0fc455868b4f32103")
	},
	{
		"bookName": "Mastering Java puzzles",
		"price": "650",
		"noOfPage": 900,
		"authorId" : ObjectId("60cae3c0fc455868b4f32103")
	},
	{
		"bookName": "Explore nNature",
		"price": "150",
		"noOfPage": 1200,
		"authorId" : ObjectId("60cae3c0fc455868b4f32104")
	}
])

 

> db.books.find().pretty()
{
	"_id" : ObjectId("60cae467fc455868b4f32105"),
	"bookName" : "Programming Puzzles",
	"price" : "450",
	"noOfPage" : 345,
	"authorId" : ObjectId("60cae3c0fc455868b4f32103")
}
{
	"_id" : ObjectId("60cae467fc455868b4f32106"),
	"bookName" : "Mastering Java puzzles",
	"price" : "650",
	"noOfPage" : 900,
	"authorId" : ObjectId("60cae3c0fc455868b4f32103")
}
{
	"_id" : ObjectId("60cae467fc455868b4f32107"),
	"bookName" : "Explore nNature",
	"price" : "150",
	"noOfPage" : 1200,
	"authorId" : ObjectId("60cae3c0fc455868b4f32104")
}

 

I want to delete the author "Rama Krishna" and all the books that he written from the database. Since this use case deals with two collections, we should go for a transaction support. If you try without transaction support, you may get data inconsistency. For example, author information may be deleted, and books are not deleted due to some hardware or business logic issue.

 

Basic steps in the transaction

There are three basic steps in the transaction flow.

a.   Begin the transaction.

b.   Execute a set of queries

c.    If no error occurs, then commit the transaction. If an error occurs, then roll back the transaction.

 

Follow below step-by-step procedure to delete the author and book documents.

 

Step 1: Start the session.

const session = db.getMongo().startSession()

 

Step 2: Start the transaction.

session.startTransaction()

Step 3: Get collection instances.

const authorsCollection = session.getDatabase("sample").authors
const booksauthorsCollection = session.getDatabase("sample").books

Step 4: Perform the operations.

authorsCollection.deleteOne({"_id" : ObjectId("60cae3c0fc455868b4f32103")})
booksauthorsCollection.deleteMany({"authorId" : ObjectId("60cae3c0fc455868b4f32103")})

Step 5: Commit the transaction.

session.commitTransaction()

Note

a. If you do not want to continue, you can abort the transaction.

session.abortTransaction()

 

b. If you try to start a session on a standalone server, you'll get this error.

Transaction numbers are only allowed on a replica set member or mongos

 

In order to use transactions, you need a MongoDB replica set

 

 

Previous                                                    Next                                                    Home

MongoDB: mongoimport: Import documents from a file

‘mongoimport’ command is used to import the documents from a file.

 

Append data to the existing collection

mongoimport {file_path_to_import} -d {database_name} -c {collection_name} --jsonArray

 

Drop the existing data from the collection and import the data from the file

mongoimport {file_path_to_import} -d {database_name} -c {collection_name} --jsonArray --drop

 

Let’s see it with an example. For example, employee collection has following data.

> db.employee.find()
{ "_id" : ObjectId("60c97d29fc455868b4f32101"), "firstName" : "Krishna", "lastName" : "Gurram", "age" : 31 }
{ "_id" : ObjectId("60c97dbcfc455868b4f32102"), "firstName" : "Siva", "lastName" : "Ponnam", "age" : 34 }

 

Now I want to append data to the employee collection from

 

emp1.json

[
	{
		"firstName" : "Chamu",
		"lastName" : "Gurram"
	},
	{
		"firstName" : "Harini",
		"lastName" : "Maj"
	}
]

 

$mongoimport /Users/Shared/mongodb/emp1.json -d sample -c employee --jsonArray
2021-06-16T15:53:48.372+0530	connected to: mongodb://localhost/
2021-06-16T15:53:48.377+0530	2 document(s) imported successfully. 0 document(s) failed to import.

Let’s query employee collection again.

> db.employee.find()
{ "_id" : ObjectId("60c97d29fc455868b4f32101"), "firstName" : "Krishna", "lastName" : "Gurram", "age" : 31 }
{ "_id" : ObjectId("60c97dbcfc455868b4f32102"), "firstName" : "Siva", "lastName" : "Ponnam", "age" : 34 }
{ "_id" : ObjectId("60c9d1349f878ae6f54110f0"), "firstName" : "Harini", "lastName" : "Maj" }
{ "_id" : ObjectId("60c9d1349f878ae6f54110f1"), "firstName" : "Chamu", "lastName" : "Gurram" }

 

Drop all the documents from employee collection and add documents from emp2.json file

emp2.json

[
	{
		"firstName" : "Venkatesh",
		"lastName" : "Battula"
	},
	{
		"firstName" : "Maruthi",
		"lastName" : "Bolisetty"
	}
]

 

$mongoimport /Users/Shared/mongodb/emp1.json -d sample -c employee --jsonArray --drop
2021-06-16T15:57:43.569+0530	connected to: mongodb://localhost/
2021-06-16T15:57:43.570+0530	dropping: sample.employee
2021-06-16T15:57:43.724+0530	2 document(s) imported successfully. 0 document(s) failed to import.

 

Let’s query employee collection again.

> db.employee.find()
{ "_id" : ObjectId("60c9d21f304ce763b168f548"), "firstName" : "Chamu", "lastName" : "Gurram" }
{ "_id" : ObjectId("60c9d21f304ce763b168f549"), "firstName" : "Harini", "lastName" : "Maj" }

 

 

 

 

 

 

 

 

 

 

Previous                                                    Next                                                    Home

MongoDB: Working with write concern

Using write concern, you can specify the level of acknowledgment requested from MongoDB for write operations.

 

Below table summarizes all the fields that are part of write concern.

 

Field	Default value	Description
w	1	Using this field, we can request acknowledgement that the write operation has propagated to a specified number of mongod instances.   w: 1 Request acknowledgement from primary mongod server.   w: 0 Request no acknowledgement from the mongod server.   w > 1 Request acknowledgment from the primary and as many data-bearing secondaries as needed to meet the specified write concern. For example, w : 3 request acknowledgment from the primary and 2 secondaries.
j	undefined or false	j:true specifies that the write operation should be written to journal.
wtimeout	0	Specifies the write timeout in milliseconds. wtimeout is only applicable for w values greater than 1.   wtimeout: 0 is equivalent to a write concern without the wtimeout option.

 

Not expecting acknowledgement from mongodb server.

> db.employee.insert({firstName : "Krishna", lastName : "Gurram", age : 31}, {writeConcern : {w: 0}})
WriteResult({ })

 

Expecting acknowledgement and write to journal

> db.employee.insert({firstName : "Siva", lastName : "Ponnam", age : 34}, {writeConcern : {j: true}})
WriteResult({ "nInserted" : 1 })

Previous                                                    Next                                                    Home

MongoDB: Ordered and unordered inserts

By default, MongoDB perform ordered inserts. Usually executing an ordered insert is slower than executing an unordered insert.

 

Ordered operations stop after an error, while unordered operations continue to process any remaining write operations in the queue.

 

Let’s see it with an example.

> db.countries.insertMany([{_id : "India", name : "India"}, {_id : "Japan", name : "Japan"}, {_id : "Iran", name : "Iran"}])
{
	"acknowledged" : true,
	"insertedIds" : [
		"India",
		"Japan",
		"Iran"
	]
}

As you see above snippet, I inserted three country names into countries collection.

 

Let’s fetch all the inserted documents.

> db.countries.find()
{ "_id" : "India", "name" : "India" }
{ "_id" : "Japan", "name" : "Japan" }
{ "_id" : "Iran", "name" : "Iran" }

Let’s insert following documents into countries collection.

[{
	_id: "China",
	name: "China"
}, {
	_id: "Japan",
	name: "Japan"
}, {
	_id: "Italy",
	name: "Italy"
}]

As you see above snippet, I am trying to insert the Japan country detail again, which leads to conflict.

> db.countries.insertMany([{_id : "China", name : "China"}, {_id : "Japan", name : "Japan"}, {_id : "Italy", name : "Italy"}])
uncaught exception: BulkWriteError({
	"writeErrors" : [
		{
			"index" : 1,
			"code" : 11000,
			"errmsg" : "E11000 duplicate key error collection: sample.countries index: _id_ dup key: { _id: \"Japan\" }",
			"op" : {
				"_id" : "Japan",
				"name" : "Japan"
			}
		}
	],
	"writeConcernErrors" : [ ],
	"nInserted" : 1,
	"nUpserted" : 0,
	"nMatched" : 0,
	"nModified" : 0,
	"nRemoved" : 0,
	"upserted" : [ ]
}) :
BulkWriteError({
	"writeErrors" : [
		{
			"index" : 1,
			"code" : 11000,
			"errmsg" : "E11000 duplicate key error collection: sample.countries index: _id_ dup key: { _id: \"Japan\" }",
			"op" : {
				"_id" : "Japan",
				"name" : "Japan"
			}
		}
	],
	"writeConcernErrors" : [ ],
	"nInserted" : 1,
	"nUpserted" : 0,
	"nMatched" : 0,
	"nModified" : 0,
	"nRemoved" : 0,
	"upserted" : [ ]
})
BulkWriteError@src/mongo/shell/bulk_api.js:367:48
BulkWriteResult/this.toError@src/mongo/shell/bulk_api.js:332:24
Bulk/this.execute@src/mongo/shell/bulk_api.js:1186:23
DBCollection.prototype.insertMany@src/mongo/shell/crud_api.js:326:5
@(shell):1:1

As per ordered inserts behavior, it stops inserting after an error, but all the documents that are inserted before error are still persisted to database.

> db.countries.find()
{ "_id" : "India", "name" : "India" }
{ "_id" : "Japan", "name" : "Japan" }
{ "_id" : "Iran", "name" : "Iran" }
{ "_id" : "China", "name" : "China" }

How can I skip the documents that cause error and proceed with other documents?

Use unordered inserts functionality. Unordered operations continue to process any remaining write operations in the queue.

 

You can enable unordered inserts by setting the property ‘order’ to false.

db.countries.insertMany([{_id : "China", name : "China"}, {_id : "Japan", name : "Japan"}, {_id : "Italy", name : "Italy"}], {ordered: false})

Let’s execute above snippet.

> db.countries.insertMany([{_id : "China", name : "China"}, {_id : "Japan", name : "Japan"}, {_id : "Italy", name : "Italy"}], {ordered: false})
uncaught exception: BulkWriteError({
	"writeErrors" : [
		{
			"index" : 0,
			"code" : 11000,
			"errmsg" : "E11000 duplicate key error collection: sample.countries index: _id_ dup key: { _id: \"China\" }",
			"op" : {
				"_id" : "China",
				"name" : "China"
			}
		},
		{
			"index" : 1,
			"code" : 11000,
			"errmsg" : "E11000 duplicate key error collection: sample.countries index: _id_ dup key: { _id: \"Japan\" }",
			"op" : {
				"_id" : "Japan",
				"name" : "Japan"
			}
		}
	],
	"writeConcernErrors" : [ ],
	"nInserted" : 1,
	"nUpserted" : 0,
	"nMatched" : 0,
	"nModified" : 0,
	"nRemoved" : 0,
	"upserted" : [ ]
}) :
BulkWriteError({
	"writeErrors" : [
		{
			"index" : 0,
			"code" : 11000,
			"errmsg" : "E11000 duplicate key error collection: sample.countries index: _id_ dup key: { _id: \"China\" }",
			"op" : {
				"_id" : "China",
				"name" : "China"
			}
		},
		{
			"index" : 1,
			"code" : 11000,
			"errmsg" : "E11000 duplicate key error collection: sample.countries index: _id_ dup key: { _id: \"Japan\" }",
			"op" : {
				"_id" : "Japan",
				"name" : "Japan"
			}
		}
	],
	"writeConcernErrors" : [ ],
	"nInserted" : 1,
	"nUpserted" : 0,
	"nMatched" : 0,
	"nModified" : 0,
	"nRemoved" : 0,
	"upserted" : [ ]
})
BulkWriteError@src/mongo/shell/bulk_api.js:367:48
BulkWriteResult/this.toError@src/mongo/shell/bulk_api.js:332:24
Bulk/this.execute@src/mongo/shell/bulk_api.js:1186:23
DBCollection.prototype.insertMany@src/mongo/shell/crud_api.js:326:5
@(shell):1:1
>

Let’s query for the countries collection.

> db.countries.find()
{ "_id" : "India", "name" : "India" }
{ "_id" : "Japan", "name" : "Japan" }
{ "_id" : "Iran", "name" : "Iran" }
{ "_id" : "China", "name" : "China" }
{ "_id" : "Italy", "name" : "Italy" }

Italy country details are persisted even though prior documents lead to conflicts.

 

Other points to remember

a.   Ordered inserts in the default behavior in MongoDB.

b.   While executing an ordered list, each operation must wait for the previous operation to finish. so executing an ordered list will generally be slower than executing an unordered list.

c.    Ordered operations stop after an error, while unordered operations continue to process any remaining write operations in the queue.

 

 

Previous                                                    Next                                                    Home