AWS Big Data Blog

Category: HAQM VPC

Stream VPC Flow Logs to Datadog via HAQM Kinesis Data Firehose

February 9, 2024: HAQM Kinesis Data Firehose has been renamed to HAQM Data Firehose. Read the AWS What’s New post to learn more. It’s common to store the logs generated by customer’s applications and services in various tools. These logs are important for compliance, audits, troubleshooting, security incident responses, meeting security policies, and many other […]

Stream VPC flow logs to HAQM OpenSearch Service via HAQM Kinesis Data Firehose

February 9, 2024: HAQM Kinesis Data Firehose has been renamed to HAQM Data Firehose. Read the AWS What’s New post to learn more. HAQM Virtual Private Cloud (HAQM VPC) flow logs enable you to track the IP traffic going to and from the network interfaces in your VPC for your workloads. Analyzing VPC logs helps […]

Simplify private network access for solutions using HAQM OpenSearch Service managed VPC endpoints

HAQM OpenSearch Service makes it easy for you to perform interactive log analytics, real-time application monitoring, website search, and more. HAQM OpenSearch is an open source, distributed search and analytics suite. HAQM OpenSearch Service offers the latest versions of OpenSearch, support for 19 versions of Elasticsearch (1.5 to 7.10 versions), as well as visualization capabilities […]

How Goldman Sachs builds cross-account connectivity to their HAQM MSK clusters with AWS PrivateLink

August 2023: HAQM MSK now offers a managed feature called multi-VPC private connectivity to simplify connectivity of your Kafka clients to your brokers. Refer this blog to learn more. This guest post presents patterns for accessing an HAQM Managed Streaming for Apache Kafka cluster across your AWS account or HAQM Virtual Private Cloud (HAQM VPC) […]

Connect to and run ETL jobs across multiple VPCs using a dedicated AWS Glue VPC

In this blog post, we’ll go through the steps needed to build an ETL pipeline that consumes from one source in one VPC and outputs it to another source in a different VPC. We’ll set up in multiple VPCs to reproduce a situation where your database instances are in multiple VPCs for isolation related to security, audit, or other purposes.