AWS Database Blog

Customize HAQM RDS events notification using HAQM EventBridge and AWS Lambda

Large customer database infrastructures are often deployed with HAQM Relational Database Service (HAQM RDS), which includes various engine types such as Oracle, SQL Server, MySQL, MariaDB, and PostgreSQL. Monitoring is an important part of maintaining the reliability, availability, and performance of HAQM RDS and your AWS solutions. To debug multi-point failures, we recommend that you collect monitoring data from all parts of your AWS solution and automate monitoring tasks as much as possible.

Monitoring database health is a mandatory requirement. You can monitor HAQM RDS databases using a variety of tools in AWS, such as HAQM RDS Performance Insights, HAQM CloudWatch, HAQM EventBridge, HAQM RDS event notification, and HAQM RDS database log files. These tools let you check the load on the database, host-level metrics, and instance activity and errors.

HAQM RDS uses the HAQM Simple Notification Service (HAQM SNS)  to provide notification when an HAQM RDS event occurs. These notifications can be in any notification form supported by HAQM SNS for an AWS Region, such as an email, a text message, or a call to an HTTP endpoint. HAQM RDS groups events into categories that you can subscribe to so that you can be notified when an event in that category occurs. You can subscribe to an event category for the DB instance, DB snapshot, DB Cluster events, DB Cluster snapshot events, and DB security group. For example, if you subscribe to the backup category for a given DB instance, you’re notified whenever a backup-related event occurs that affects the DB instance. If you subscribe to a configuration change category for a DB security group, you’re notified when the DB security group is changed. You also receive notification when an event notification subscription changes. You might want to create several different subscriptions. For example, you might create one subscription receiving all event notifications and another subscription that includes only critical events for your production DB instances.

In this post, we walk through the steps to set up HAQM RDS event notification using an EventBridge rule and AWS Lambda by customizing the event notifications for specific events, along with sending custom messages to operation teams and interested parties using HAQM Simple Notification Service (HAQM SNS) HAQM Simple Queue Service (HAQM SQS).

Solution overview

HAQM RDS events offers various categories of events that are generated when specific database events occur. Operations teams that support database environments must be notified of any major database events like shutdowns, restarts or failovers. This helps database administrators take proper actions as required. To make the events accessible and to tightly integrate with third-party monitoring, the solution creates/uses generalized JSON-formatted alert and event message data is sent to those systems to process further actions.

The solution in this post enables you to do the following:

  • Send custom JSON-formatted event notifications to HAQM SQS, and integrate email subscriptions with third-party monitoring systems
  • Notify end-users with email notifications of any occurrence of specific HAQM RDS events instead of monitoring all the events in the same class
  • Customize specific databases that you want to check and SNS topics that you want to send notifications of

The following diagram illustrates our solution architecture.

The provided AWS CloudFormation template creates the following resources:

  • An SNS topic and AWS Identity and Access Management (IAM) policy
  • HAQM SNS subscriptions for email and HAQM SQS
  • An IAM role and policies for running Lambda functions
  • A Lambda function and EventBridge rule for HAQM RDS events notification

Note: You will be billed for the AWS resources used if you create a stack from this template.

We walk you through each step in the template, and demonstrate how to deploy and test the solution.

Pre-requisites

To deploy this solution, first complete the following steps:

  1. Create and activate an AWS account or use an existing account if you have one already..
  2. Create an HAQM RDS instance or use an existing instance if you have one already.

Deploy the solution using AWS CloudFormation

This section demonstrates the process of setting up the resources needed for this solution using the CloudFormation template. The stack creates the SNS topic, SNS subscriptions, IAM role, and Lambda function, and includes a sample Python code to send notifications and an EventBridge rule to invoke the function whenever an HAQM RDS event occurs.

  1. To get started, choose Launch Stack.
  2. For Stack name, enter a name.
  3. Enter a RDS Identifier to monitor and an email address for sending the SNS alerts. Additional Subscription required on the SNS topic for each email address.
  4. Choose Next.
  5. Choose Next twice.
  6. Choose I acknowledge that AWS CloudFormation might create IAM resources.
  7. Choose Create stack.

The stack takes about five minutes to deploy. When it’s complete, your customized HAQM RDS events monitoring and notification process starts.

Test HAQM RDS event notifications

To test the solution, complete the following steps:

  1. On the HAQM RDS console, choose Databases.
  2. Choose the database you want to monitor.
  3. On the Actions menu, choose an action that generates the HAQM RDS event and notification (for example, choose Stop to stop the database)

Note: Choosing Stop action will stop the database and make it unavailable to users.

After about three minutes, you should receive an email alert with details in JSON format.

You can check the SQS queue for the message via the HAQM SQS console.

Solution Components

HAQM SNS topic

HAQM SNS is a fully managed messaging service for both application-to-application (A2A) and application-to-person (A2P) communication. The CloudFormation template creates a new standard SNS topic and IAM policy for the SNS topic, which grants a list of HAQM SNS related privileges. Make sure that your organization infosec standards are not violated. For information about HAQM Simple Notification Service endpoints and quotas.

HAQM SNS subscriptions

To receive messages published to a topic, you must subscribe an endpoint to the topic. When you subscribe an endpoint to a topic, the endpoint receives messages published to the associated topic. The following sections walk through the steps involved in creating email and HAQM SQS endpoint types as part of creating an SNS subscription. 

Email subscription

To receive email alerts, the CloudFormation template creates the email subscription on the SNS topic created in the previous step, following the stack’s dynamic parameter pSNSEmail value. The subscription sends the alert in JSON-formatted text to the email subscriber. You will receive an email to confirm the subscription after you create the stack, which you must do to activate the HAQM SNS email subscription.

HAQM SQS subscription

HAQM SQS is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications. For more details , check the HAQM SQS FAQs and HAQM SQS quota. You can use any third-party monitoring tools to read messages from HAQM SQS and process them. The CloudFormation template creates the SQS queue and subscription on the SNS topic created in the previous step. The custom JSON-formatted event information is sent via HAQM SQS to third-party monitoring systems for further integrations.

IAM role for Lambda

An IAM role is an IAM identity that you can create in your account that has specific permissions. An IAM role is similar to an IAM user, in that it’s an IAM policy that decides what the identity can and can’t do in AWS. In this example, The CloudFormation template creates an IAM role which grants specific permissions to run Lambda functions and invoke HAQM SNS.

Because this CloudFormation stack deployment includes IAM role creation, during the stack deployment you need to acknowledge that AWS CloudFormation will create IAM resources. You should use least privilege as needed for this role or per your info-sec requirements after a careful review of the permissions. The following code is a snippet from the CloudFormation template and shows the IAM role policy document.

        "rRole": {
            "Type": "AWS::IAM::Role",
            "Properties": {
                "AssumeRolePolicyDocument": {
                    "Statement": [
                        {
                            "Effect": "Allow",
                            "Principal": {
                                "Service": [
                                    "lambda.amazonaws.com"
                                ]
                            },
                            "Action": [
                                "sts:AssumeRole"
                            ]
                        }
                    ]
                },
                "ManagedPolicyArns": [
                    "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole",
                    "arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess"
                ],
                "Policies": [
                    {
                        "PolicyName": "MonitoringEventsCriticalSNSPolicy",
                        "PolicyDocument": {
                            "Version": "2012-10-17",
                            "Statement": [
                                {
                                    "Action": [
                                        "sns:ListTagsForResource",
                                        "sns:Publish",
                                        "sns:GetTopicAttributes"
                                    ],
                                    "Resource": [
                                        {
                                            "Fn::Sub": "arn:aws:sns:${AWS::Region}:${AWS::AccountId}:MonitoringEventsCritical"
                                        }
                                    ],
                                    "Effect": "Allow"
                                },
                                {
                                    "Action": [
                                        "sns:ListPhoneNumbersOptedOut",
                                        "sns:GetEndpointAttributes",
                                        "sns:GetPlatformApplicationAttributes",
                                        "sns:GetSubscriptionAttributes",
                                        "sns:GetSMSAttributes",
                                        "sns:CheckIfPhoneNumberIsOptedOut"
                                    ],
                                    "Resource": [
                                        {
                                            "Fn::Sub": "arn:aws:sns:${AWS::Region}:${AWS::AccountId}:MonitoringEventsCritical"
                                        }
                                    ],
                                    "Effect": "Allow"
                                }
                            ]
                        }
                    }
                ],
                "Description": "IAM role to provide needed permission for Lambda function to execute and store logs in cloudwatch"
            }
        }
                "Description": "IAM role to provide needed permission for Lambda function to execute and store logs in cloudwatch"
            }
        },

Lambda function and EventBridge rule

As a last step, we create a Lambda function with specific configurations and an EventBridge rule to call that Lambda function. You can verify the function on the Lambda console after deployment. The following are the Lambda environment variables:

  • RDSEventID – A comma-separated list of specific HAQM RDS event IDs.
  • RDSIdentifier – A comma-separated list of RDS database identifiers, which are required to be part of monitoring. A dynamic input parameter is mapped to collect the value during stack creation.
  • SNSTopic – An SNS topic to send notifications.

For all HAQM RDS events, the event rule triggers the Lambda function. The function processes the events by filtering based on the RDSEventID and RDSIdentifier environment variables. It also composes email content in JSON format to send notifications via email and the SQS queue. For more information about HAQM RDS events, see HAQM RDS event categories and event messages.

You can customize RDSEventID by updating a specific event ID list in the Lambda environment variable of your choice to get notified. You can edit the Lambda environment variable RDSEventID with specific events before deploying the CloudFormation stack, and you can also edit post-deployment by navigating to the Lambda console and updating the respective function input variable as needed. Make sure to remediate CloudFormation stack drift if you update the value through the console.

We also create a CloudWatch log group as part of this Lambda function to make sure it’s logging the details of each function run which is useful for any debugging. The following code creates the log group:

        "rRDSEventNotificationLogGroup": {
            "Type": "AWS::Logs::LogGroup",
            "DependsOn": "rRDSEventNotification",
            "DeletionPolicy": "Delete",
            "Properties": {
              "LogGroupName": {"Fn::Join": ["", ["/aws/lambda/", {"Ref": "rRDSEventNotification"}]]},
              "RetentionInDays": 14
            }
          }

After creating the function, the template creates the EventBridge rule that can trigger the function during any HAQM RDS events.
The following is the CloudFormation stack code that creates the EventBridge rule and enables it:

"rRDSEventNotificationRuleblog": {
            "Type": "AWS::Events::Rule",
            "Properties": {
                "Description": "To call Lambda to send custom notification using sns topic whenever RDS events occur",
                "Name": "RDSEventNotificationRuleblog",
                "EventPattern": {
                    "source": [
                        "aws.rds"
                    ],
                    "detail-type": [
                        "RDS DB Instance Event",
                        "RDS DB Security Group Event",
                        "RDS DB Parameter Group Event",
                        "RDS DB Snapshot Event",
                        "RDS DB Cluster Event",
                        "RDS DB Cluster Snapshot Event"
                    ]
                },
                "State": "ENABLED",
                "Targets": [
                    {
                        "Arn": {
                            "Fn::GetAtt": [
                                "rRDSEventNotification",
                                "Arn"
                            ]
                        },
                        "Id": "RDSEventNotification"
                    }
                ]
            }
        }

Clean up

To avoid incurring ongoing charges, delete the resources created as part of the CloudFormation template when you’re done using the solution.

Conclusion

In this post, we demonstrated how to customize HAQM RDS events notification using CloudWatch and Lambda, and used AWS CloudFormation to automate this setup. Monitoring databases is critical for every organization; with this solution, we can monitor databases using HAQM RDS events, send notifications to the operations team, and integrate with third-party monitoring systems using custom JSON-formatted alert messages. This helps you manage and watch all your HAQM RDS database environments. You can also add and remove required HAQM RDS events based on your requirements.

Try out the solution for yourself, and leave your thoughts in the comments.

About the Authors

Suresh Kumar Gopalan is a Database Consultant with AWS Professional Services, Strategic Practice. He has deep expertise in SQL/NoSQL database technologies. He has developed many successful database solutions that addressed challenging on-premises database migration to AWS. He has extensively worked in heterogeneous migrations, performance tuning, automation using TSQL and PowerShell, Oracle Golden Gate, and SQL Server P2P replication with active-active implementation.

Siva Subramaniam is a Team Lead-Database Specialty with AWS Professional Services, US West Application Practice, and has two decades of IT experience in technical leadership and application and database design. He also has hands-on experience as an infrastructure and application support DBA, including database architecture, design, build, data modeling, infrastructure automation, migrations and upgrades, performance tuning, monitoring, backup and recovery, and disaster recovery. Siva holds a master’s degree in Computer Applications and is a certified professional in various database and cloud technologies including AWS, Azure, OCI, Oracle, MySQL, Cassandra, and VMware. Siva enjoys playing cricket with his friends and son, doing farming and agriculture-related activities, and learning to cook from his wife.