AWS Cloud Operations Blog

Detect and respond to security threats in near real-time using HAQM Managed Grafana

Security is “job zero” at AWS. It’s crucial to gain deeper insights into your AWS infrastructure’s security posture to respond quickly to threats. The ability to centrally monitor and visualize the security findings make it easier for you to identify any security threats or gaps and also keep the principle of least privilege in focus. You can visualize the insights into those security findings across multiple accounts using HAQM Managed Grafana that retrieves and refreshes the data periodically.

AWS Security Hub offers a comprehensive overview of your AWS security status, allowing you to compare it with industry standards and best practices. Organizations may have additional requirements to centralize AWS Security Hub findings and integrate them with other operational data. Some of the benefits include consolidating AWS Security Hub findings across regions into a single dashboard view, and centralizing and correlating various security & compliance data along with operational data into one dashboard.

In this post we will show you how you can centralize your AWS Security Hub findings, that provide a single-pane-of-glass on workloads running on AWS cloud using HAQM Managed Grafana.

We will demonstrate similar integration with HAQM GuardDuty and HAQM Inspector through another post.

Architecture Overview:

Figure 1: Architecture Overview

Please refer to the details on components of the solution depicted in the above architecture:

  1. AWS Security Hub – AWS Security Hub is a cloud security posture management service that performs security best practices checks, aggregates alerts, and enables automated remediation.
  2. HAQM EventBridge Rule – HAQM EventBridge rule will be triggered on a new finding or an update to existing finding in AWS Security Hub.
  3. Custom AWS Lambda to process the Event – An AWS Lambda function to extract and transform the complex nested AWS Security Hub finding to a simpler JSON format that is more suitable to perform further analytics.
  4. HAQM Simple Storage Service (HAQM S3) bucket to receive the processed event – The AWS Lambda function delivers each finding to this HAQM S3 bucket.
  5. AWS Glue crawler and AWS Glue Data Catalog – Configure an AWS Glue crawler to run every hour to parse the JSON data stored in HAQM S3 and update the AWS Glue Data Catalog containing the JSON table schema, metadata, and database partitions. This metadata is required to perform queries against the data in HAQM S3.
  6. HAQM Athena Workgroup – HAQM Athena is used to look at the schema from the AWS Glue Data Catalog and query the data in S3. The HAQM Managed Grafana Dashboard must have permissions to execute queries against the HAQM S3 data using HAQM Athena Workgroup and AWS Glue Data Catalog.
  7. HAQM Managed Grafana – Deploy a dashboard using HAQM Managed Grafana to visualize data.

Using the AWS CloudFormation template provided in this blog, deploy the required resources in the same AWS Region and AWS account containing the Security Hub service where findings of all regions and accounts are aggregated into.

Prerequisites:

  1. Enable AWS Security Hub in member accounts. Enable required security standards, AWS native service integration, and partner integrations in all the member accounts across your AWS Regions.
  2. Set up your AWS Security Hub administrator account. Designate one of the AWS accounts within your AWS Organizations to be a delegated administrator for AWS Security Hub. This account can manage and receive findings across member accounts.
  3. Enable Cross Region Aggregation.
  4. Set up HAQM Athena workgroups.
  5. Set up HAQM Managed Grafana workspace. For information, and steps for creating the HAQM Managed Grafana workspace, see Creating a Workspace.

Once you have all the prerequisites in place, follow the instructions below for visualizing AWS Security Hub findings on HAQM Managed Grafana.

Deployment Steps:

Step 1: Launch the AWS CloudFormation template

Download and launch this AWS CloudFormation template to deploy AWS Lambda, HAQM S3 Bucket, AWS Glue Crawler, AWS Glue Database and its related components.

Note: Some of the resources that this stack deploys incur costs when in use.

Follow these steps to generate your resources utilizing an AWS CloudFormation template:

  1. Sign in to the AWS Management Console.
  2. Navigate to the AWS CloudFormation console > Create Stack > “With new resources”.
  3. Upload the yaml template file and choose Next.
  4. Specify a “Stack name” and choose Next.
  5. Leave the “Configure stack options” at default values and choose Next.
  6. Review the details on the final screen and under “Capabilities” check the box for “I acknowledge that AWS CloudFormation might create IAM resources with custom names”.
  7. Choose Submit.


Figure 2: Acknowledgement

Note: You can review the progress of your new stack under AWS CloudFormation > Stacks > [StackName] > Events tab

Once the Stack is created successfully, you will see the following resources deployed: HAQM EventBridge scheduler, AWS Lambda Function, HAQM S3 Bucket, AWS Glue Crawler, AWS Glue Database and the corresponding AWS IAM Roles and Policies are created successfully.

Step 2: Create View in HAQM Athena using the below queries created as part of the AWS CloudFormation stack

  1. Go to HAQM Athena > Query editor > Saved queries tab and choose the query named “AWS-securityhub”.
    Note: Workgroup created is named “Primary”

Figure 3: HAQM Athena Saved Queries

  1. On the Query editor, verify the Data source, Database and Table names while running the query. Upon successful execution, the query creates a View named “security_hub_findings”.

Figure 4: HAQM Athena Query Editor

Step 3: Configure HAQM Athena Data Source in HAQM Managed Grafana

  1. Access the HAQM Managed Grafana console through the provided HAQM Managed Grafana workspace URL and log in using the user credentials you’ve set up.
  2. Navigate to Administration > Data sources and select HAQM Athena from the options.
  3. Adjust the HAQM Athena settings by specifying the Default Region (us-east-1), Data source (AWSDataCatalog), Database (aws-security-hub-db), Workgroup (primary), and set the Output Location for your HAQM Athena query.
  4. Choose “Save & test” to confirm that the data source is functioning properly. You can now begin querying and visualizing metrics from the AWS environment.
    Note: If you encounter a permission denied error, ensure that the HAQM Managed Grafana service role permissions, as discussed in the previous step, are correctly configured.

Figure 5: HAQM Athena as Data source

Step 4: Create an HAQM Managed Grafana Dashboard

HAQM Managed Grafana is a fully managed service designed to simplify the process of creating, configuring, and sharing interactive dashboards and charts for monitoring your data. It offers the ability to establish alerts and notifications based on specific conditions or thresholds, enabling swift identification and response to issues.

In this next step, we will utilize HAQM Managed Grafana to generate a new AWS Security Hub findings detection dashboard.

  1. Retrieve the AWS Security Hub findings dashboard JSON file from this link.
  2. Import the dashboard by navigating to Dashboards > New and selecting Import in the HAQM Managed Grafana console. For additional information on exporting and importing dashboards, refer to the documentation.

Figure 6: HAQM Managed Grafana Dashboard

Finally, AWS Security Hub findings are integrated into HAQM Managed Grafana. This dashboard updates every 5 minutes, querying the materialized views established in HAQM Athena.

Furthermore, HAQM Managed Grafana’s alerting system delivers strong and actionable alerts, enabling us to swiftly identify system issues as soon as they arise. For further insights into HAQM Managed Grafana alerting, please visit the “Alerts in HAQM Managed Grafana” section.

Clean up

To avoid incurring future charges, delete all resources used in this post.

In this blog post, we showed how you can visualize and analyze your AWS Security Hub with HAQM Managed Grafana. By identifying potential threats quickly, sensitive data can be safeguarded more effectively. Near real-time dashboards allow for proactive measures, ensuring that critical workload remains secure.

To learn more and get hands-on experience on AWS observability services, check the One Observability Workshop.

About the authors:

Sameeksha Garg

Sameeksha is a Technical Account Manager at AWS committed to accelerate the cloud journey for AWS Global Enterprise customers. She has 7+ years of industry experience across cloud security, cloud operations, cloud infrastructure management and customer advocacy. She is passionate about cloud security technologies and strives to help customers secure their workloads in the cloud.

Anjali Sharma

Anjali Sharma is a Technical Account Manager (TAM) at AWS with more than 7 years of IT experience. Her diverse career includes roles such as Cloud Consultant and Operations Engineer at AWS Managed Services. In her current position, she collaborates with global customers to develop sustainable software solutions. She has a passion for troubleshooting and enhancing operational excellence for her customers.

Yash Bindlish

Yash is an Enterprise Support Manager at HAQM Web Services. He has more than 17 years of industry experience including roles in cloud architecture, systems engineering, and infrastructure. He works with Global Enterprise customers and help them build, scalable, modern and cost effective solutions on their growth journey with AWS. He loves solving complex problems with his solution-oriented approach.

Scott Kellish

Scott Kellish is a Sr Partner Solutions Architect / WW Tech Lead for Cloud Operations., with 8+ years at AWS, working with AWS Partners and customers on all things related to cloud operations. Outside of work, he enjoys working outside, exploring new places by bicycle with his wife and sitting on a beach in Nantucket being thankful for life.