Optimizing costs on HAQM DocumentDB using event-driven architecture and the AWS EventBridge Terraform module

A primary reason companies move their workloads to AWS is because of cost. With AWS, cloud migration and application modernization plans are based on your business needs and not agreements or licensing. You can acquire technology on an as-needed basis, only paying for the resources you use. You can build modern, scalable applications on AWS to transform your organization, all while optimizing costs.

In this post, we discuss how to optimize costs on HAQM DocumentDB (with MongoDB compatibility) using an event-driven architecture with AWS Lambda, HAQM EventBridge, and the AWS EventBridge Terraform module by automating the starting and stopping of clusters around business hours.

Cost-optimization

HAQM DocumentDB recently announced I/O-Optimized pricing, a new storage configuration that provides improved price-performance and predictable pricing for customers with I/O-intensive applications. If you’re running an HAQM DocumentDB cluster with Standard storage configuration, you can run the HAQM DocumentDB Deployment Scanner to estimate possible cost-optimization and utilization by moving to the I/O-Optimized storage configuration.

Also available is document compression for your HAQM DocumentDB 5.0 instance-based clusters. Through use of the LZ4 compression algorithm, compressed documents are up to 7x smaller than uncompressed documents. Smaller documents require less storage space and IO operations, leading to lower storage and IO costs.

In addition to these features, there is more you can do to recognize cost savings on your HAQM DocumentDB workloads. Much like turning off the lights in your home when you’re not there, HAQM DocumentDB allows you to temporarily stop instances in your cluster when they aren’t needed, and start them again when you’re ready to resume. While your cluster is stopped, the cluster storage volume remains unchanged. You are charged only for storage, manual snapshots, and automated backup storage within your specified retention window. You aren’t charged for any instance hours.

As an example, an organization is running a development HAQM DocumentDB environment with a three-node cluster, 24 hours per day. If they stopped the instances on this cluster during nights and weekends, then turned them back on during working business hours, this would result in a reduction of over 73% in total on-demand instance cost per month, as illustrated in the following table.

Table 1: Cost Illustrations

Table 2: Pricing Assumptions

You can enhance this agility through automation with tools like Terraform that give you the ability to deploy architectures that automatically schedule resource provisioning to match the demand of your business and workload.

Automation

Using predefined tags to automate the managing and administering of AWS resources is available in services like HAQM Relational Database Service (HAQM RDS) and HAQM Elastic Compute Cloud (HAQM EC2). For more information, refer to Save costs by automating the start and stop of HAQM RDS instances with AWS Lambda and HAQM EventBridge and Automate starting and stopping AWS instances, respectively.

To help you implement a similar solution for your HAQM DocumentDB resources, we provide an overview of the Terraform project created by the HAQM DocumentDB team that uses the AWS EventBridge Terraform module.

Solution overview

You can download this Terraform project from the amazon-documentdb-samples GitHub repository and run against your HAQM DocumentDB clusters in minutes.

The solution uses two Python scripts run in Lambda: one to stop the instances in a cluster based on the AutoStop tag, and the other to start them based on the AutoStart tag.

Figure 1: Tag Details

This feature uses HAQM EventBridge Scheduler through the templated target for Lambda that calls the Invoke operation by providing each Lambda function’s ARN. The following screenshot shows an example schedule for the AutoStop function.

Figure 2: AutoStop Schedule

The following screenshot shows an example schedule for the AutoStart function.

Figure 3: AutoStart Schedule

This is designed to stop your appropriately tagged HAQM DocumentDB clusters every weekday (Monday–Friday) at 5:30PM UTC and start them again every weekday (Monday–Friday) at 8:00 AM UTC.

All of these services use HAQM CloudWatch to capture logs that you can use for diagnostics or progress monitoring, as shown in the following screenshot.

Figure 4: CloudWatch Logs

Summary

As a fully managed document database, HAQM DocumentDB increases productivity by providing improved performance, scalability, and availability without having to worry about managing the underlying infrastructure. Whether you are migrating an existing application to HAQM DocumentDB or building a new workload, you want to make sure your financial resources are being used properly and you’re getting the most out of the service. Establishing cost-optimization strategies early on maximizes your resource utilization, achieves business outcomes at the lowest possible price point, and meets your functional requirements.

By pairing tools like Terraform with AWS services like EventBridge and Lambda, you can reduce your costs by turning off your HAQM DocumentDB clusters when they are not in use. Refer to the amazon-documentdb-samples GitHub repository to try the solutions discussed in this post.

About the Author

Sonu Kumar Singh is an AWS Solutions Architect, with a specialization in analytics domain. He has been instrumental in catalyzing transformative shifts in organizations by enabling data-driven decision-making thereby fueling innovation and growth. He enjoys it when something he designed or created brings a positive impact. At AWS his intention is to help customers extract value out of AWS’s 200+ cloud services and empower them in their cloud journey.

Cody Allen is a Principal DocumentDB Specialist Solutions Architect based out of Texas. He is passionate about working side by side with customers to solve complex problems, and supporting teammates through mentorship and knowledge transfer. He has spent his career deploying and managing systems, software, and infrastructure for B2B SaaS providers, materiel and logistics suppliers, the U.S. Air Force, and other domestic and international government agencies.