AWS Database Blog

Introducing MongoDB 4.0 compatibility and Transactions in HAQM DocumentDB

On March 2023 AWS announced added support for MongoDB 5.0 drivers with HAQM DocumentDB 5.0.

HAQM DocumentDB (with MongoDB compatibility) is a fast, scalable, highly available, and fully managed document database service that supports MongoDB workloads. Today we’re announcing compatibility with MongoDB 4.0 for HAQM DocumentDB. With this launch, you can now use atomic, consistent, isolated, and durable (ACID) transactions, open a change stream cursor for a database or cluster, and much more. For the full release notes for HAQM DocumentDB 4.0, see MongoDB 4.0 Compatibility.

In this post, I summarize what’s new in HAQM DocumentDB 4.0 and show you how to get started with HAQM DocumentDB 4.0 and transactions using an AWS Cloud9 environment.

What’s new in HAQM DocumentDB 4.0?

The following are some of the major features and capabilities that were introduced in HAQM DocumentDB 4.0. To see a full list of the new capabilities, see MongoDB 4.0 Compatibility.

  • ACID Transactions – HAQM DocumentDB now supports the ability to perform transactions across multiple documents, statements, collections, and databases. Transactions simplify application development by enabling you to perform ACID operations across one or more documents within an HAQM DocumentDB cluster. For more information, see Transactions.
  • Change streams – You can now open a change stream at the cluster level (client.watch() or mongo.watch()) and the database level (db.watch()). You can also specify a startAtOperationTime to open a change stream cursor, and extend your change stream retention period to 7 days (previously, the limit was 24 hours). For more information, see Using Change Streams with HAQM DocumentDB.
  • AWS DMS – You can now use AWS Database Migration Service (AWS DMS) to migrate your MongoDB 4.0 workloads to HAQM DocumentDB. AWS DMS now supports a MongoDB 4.0 source, HAQM DocumentDB 4.0 target, and an HAQM DocumentDB 3.6 source for performing upgrades between HAQM DocumentDB 3.6 and 4.0. For more information, see Using HAQM DocumentDB as a target for AWS Database Migration Service.
  • Monitoring – With the addition of transactions, you can now monitor your transaction usage with five new HAQM CloudWatch metrics: TransactionsOpen, TransactionsOpenMax, TransactionsAborted, TransactionsStarted, and TransactionsCommitted, in addition to new fields in currentOp, ServerStatus, and profiler. For more information, see Monitoring HAQM DocumentDB with CloudWatch.
  • Performance and indexing – Included in this release are multiple performance and indexing improvements: the ability to use an index with the $lookup aggregation stage, find() queries with projections can be served directly from an index (covered query), the ability to use hint() with the findAndModify API, performance optimizations for $addToSet operator, and improvements to reduce overall index sizes. For more information, see Release Notes.
  • Operators – We have added support for new aggregation operators: $ifNull, $replaceRoot, $setIsSubset, $setInstersection, $setUnion, and $setEquals. For more information, see Supported MongoDB APIs, Operations, and Data Types.
  • Role based access control (RBAC) – With the ListCollection and ListDatabase commands, you can now optionally use the authorizedCollections and authorizedDatabases parameters to allow users to list the collections and databases that they have permission to access without requiring the listCollections and listDatabase roles, respectively. Users can also end their own cursors without requiring the KillCursor role. For more information, see Restricting Database Access Using Role-Based Access Control (Built-In Roles).

Getting started with HAQM DocumentDB 4.0 and transactions

The first step is to create an AWS Cloud9 environment and an HAQM DocumentDB cluster in your default HAQM Virtual Private Cloud (HAQM VPC). For instructions on creating a default VPC, see Getting Started with HAQM VPC. This post demonstrates how to connect to your HAQM DocumentDB cluster from your AWS Cloud9 environment with a mongo shell and run a transaction. When creating AWS resources, we recommend that you follow the best practices for AWS Identity and Access Management (IAM).

The following diagram shows the final architecture of this walkthrough.

For this walkthrough, use the default VPC in a given Region. For more information, see Creating a Virtual Private Cloud (VPC).

Creating an AWS Cloud9 environment

To create your AWS Cloud9 environment, complete the following steps:

  1. On the AWS Cloud9 console, choose Create environment.
  2. Under Environment name and description, for Name, enter a name for the environment. This post enters the name DocumentDBCloud9.
  3. Choose Next step.
  4. In the Configure settings section, accept all defaults.
  5. Choose Next step.
  6. In the Review section, choose Create environment.

The provisioning of the AWS Cloud9 environment can take up to 3 minutes. When it’s complete, you see a command prompt.

You’re redirected to the command prompt to install the mongo shell and connect to your HAQM DocumentDB cluster.

Creating a security group

In this step, you use HAQM Elastic Compute Cloud (HAQM EC2) to create a new security group that enables you to connect to your HAQM DocumentDB cluster on port 27017 (the default port for HAQM DocumentDB) from your AWS Cloud9 environment.

  1. On the HAQM EC2 console, under Network & Security, choose Security groups.
  2. Choose Create security group.
  3. For Security group name, enter demoDocDB.
  4. For VPC, accept the usage of your default VPC.
  5. For Description, enter a description.
  6. In the Inbound rules section, choose Add rule.
  7. For Type, choose Custom TCP Rule.
  8. For Port Range, enter 27017.The source security group is the security group for the AWS Cloud9 environment you just created.
  9. To see a list of available security groups, enter cloud9 in the destination field.
  10. Choose the security group with the name aws-cloud9-<environment name>.
  11. Accept all other defaults and choose Create security group.

The following screenshot shows you the security groups that were created in this step and the AWS Cloud9 security group that was created when you created an AWS Cloud9 environment.

Creating an HAQM DocumentDB 4.0 cluster

To create your HAQM DocumentDB 4.0 cluster, complete the following steps:

  1. On the HAQM DocumentDB console, on the Clusters page, choose Create.
  2. For Engine version, choose the default (4.0.0).
  3. On the Create HAQM DocumentDB cluster page, for Instance class, choose t3.medium.
  4. For Number of instances, choose 1.
    This helps minimize costs.
  5. Leave other settings at their default.
  6. In the Authentication section, enter a username and password.
  7. Turn on Show advanced settings.
  8. In the Network settings section, for VPC security groups, choose demoDocDB.
  9. Choose Create cluster.

HAQM DocumentDB is now provisioning your cluster, which can take up to a few minutes to finish. You can connect to your cluster when both the cluster and instance status show as Available. While HAQM DocumentDB provisions the cluster, complete the remaining steps to connect to your HAQM DocumentDB cluster.

Installing the 4.0 mongo shell

You can now install the mongo shell, which is a command-line utility that you use to connect to and query your HAQM DocumentDB cluster.

  1. On the AWS Cloud9 console, under Your environments, choose DocumentDBCloud9.
  2. Choose Open IDE.
  3. To install the 4.0 mongo shell, at the command prompt, create the repository file with the following code: 
    echo -e "[mongodb-org-4.0] \nname=MongoDB Repository\nbaseurl=http://repo.mongodb.org/yum/amazon/2013.03/mongodb-org/4.0/x86_64/\ngpgcheck=1 \nenabled=1 \ngpgkey=http://www.mongodb.org/static/pgp/server-4.0.asc" | sudo tee /etc/yum.repos.d/mongodb-org-4.0.repo
  4. When it’s complete, install the mongo shell with the following code: 
    sudo yum install -y mongodb-org-shell

    Transport Layer Security (TLS) is enabled by default for any new HAQM DocumentDB clusters. For more information, see Managing HAQM DocumentDB Cluster TLS Settings.

  5. To encrypt data in transit, download the CA certificate for HAQM DocumentDB. See the following code: 
    wget http://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem

Connecting to your HAQM DocumentDB cluster

You’re now ready to connect to your HAQM DocumentDB cluster.

  1. On the HAQM DocumentDB console, on the Clusters page, locate your cluster. This post uses the cluster docdb-2020-10-09-21-45-11.
  2. Choose the cluster you created.
  3. Copy the connection string provided. Omit <insertYourPassword> so that you’re prompted for the password by the mongo shell when you connect. This way, you don’t have to type your password in cleartext.Your connection string should look like the following screenshot.
  4. When you enter your password and can see the rs0:PRIMARY> prompt, you’re successfully connected to your HAQM DocumentDB cluster.

For information about troubleshooting, see Troubleshooting HAQM DocumentDB.

When you have connected with the mongo shell, you can discover the version (4.0.0) with the following command:

db.version()

You get the following output:

4.0.0

Using transactions

Now that you’re connected to your cluster with the mongo shell, you can explore using transactions. One of the classic use cases for transactions is debiting money from one person’s account and crediting that money in another person’s account. Because the use case deals with two separate operations in the database, it’s desirable that the two operations run within a transaction and follow the ACID properties. For this post, we transfer $400 from Bob’s bank account to Alice’s bank account. Both accounts begin with $500.

  1. To start from with an empty collection, first drop the account collection: 
    db.account.drop()

    You get the following output:

    {true, false}
  2. Insert data into the collection to represent Bob’s account: 
    db.account.insert({"_id": 1, "name": "Bob", "balance": 500.00});

    You get the following output:

    WriteResult({ "nInserted" : 1 })
  3. Insert data into the collection to represent Alice’s account: 
    db.account.insert({"_id": 2, "name": "Alice", "balance": 500.00});

    You get the following output:

    WriteResult({ "nInserted" : 1 })
  4. To start a transaction, create a session and a session object for the account: 
    var mySession = db.getMongo().startSession();
var mySessionObject = mySession.getDatabase('test').getCollection('account');
mySession.startTransaction({readConcern: {level: 'snapshot'}, writeConcern: {w: 'majority'}});
  5. Within the transaction, debit $400 from Bob’s account: 
    mySessionObject.updateOne({"_id": 2}, {"$inc": {"balance": 400}});

    You get the following output:

    { "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 1 }
  6. Similarly, credit Alice’s account with $400: 
    mySessionObject.updateOne({"_id": 1}, {"$inc": {"balance": -400}});

    You get the following output:

    { "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 1 }
  7. Within the transaction, you can see both updates with the following code: 
    mySessionObject.find()

    You get the following output:

    { "_id" : 2, "name" : "Alice", "balance" : 900 }
{ "_id" : 1, "name" : "Bob", "balance" : 100 }
  8. If you view outside of the transaction, the updates aren’t yet visible: 
    db.account.find()

    You get the following output:

    { "_id" : 1, "name" : "Alice", "balance" : 500 }
{ "_id" : 2, "name" : "Bob", "balance" : 500 }
  9. Commit the transaction and end the session: 
    mySession.commitTransaction()
mySession.endSession()
  10. To see the updates, enter the following code: 
    db.account.find()

    You get the following output:

    { "_id" : 2, "name" : "Alice", "balance" : 900 }
{ "_id" : 1, "name" : "Bob", "balance" : 100 }

Cleaning up

When you complete the walkthrough, you can either stop your HAQM DocumentDB cluster to reduce costs or delete the cluster. By default, after 30 minutes of inactivity, your AWS Cloud9 environment stops the underlying EC2 instance to help save costs.

Summary

This post introduced you to MongoDB 4.0 compatibility in HAQM DocumentDB and showed you how to get started with HAQM DocumentDB 4.0 and transactions by creating an AWS Cloud9 environment, installing the mongo 4.0 shell, creating an HAQM DocumentDB cluster, connecting to your cluster, and walking through a common use case for transactions. For more information, see MongoDB 4.0 Compatibility and Transactions. For more information about recent launches and blog posts, see HAQM DocumentDB (with MongoDB compatibility) resources.

About the author

 

Joseph Idziorek is a Principal Product Manager at HAQM Web Services.