AWS Big Data Blog

Tag: HAQM MSK

Break data silos and stream your CDC data with HAQM Redshift streaming and HAQM MSK

Data loses value over time. We hear from our customers that they’d like to analyze the business transactions in real time. Traditionally, customers used batch-based approaches for data movement from operational systems to analytical systems. Batch load can run once or several times a day. A batch-based approach can introduce latency in data movement and […]

Stream Twitter data into HAQM Redshift using HAQM MSK and AWS Glue streaming ETL

This post demonstrates how customers, system integrator (SI) partners, and developers can use the serverless streaming ETL capabilities of AWS Glue with HAQM Managed Streaming for Kafka (HAQM MSK) to stream data to a data warehouse such as HAQM Redshift. We also show you how to view Twitter streaming data on HAQM QuickSight via HAQM Redshift.

Vortexa delivers real-time insights on HAQM MSK with Lenses.io

This post discusses how Vortexa harnesses the power of Apache Kafka to improve real-time data accuracy and accelerate time-to-market by using a combination of Lenses.io for greater observability and HAQM Managed Streaming for Apache Kafka (HAQM MSK) to create clusters on demand.

Best practices from Delhivery on migrating from Apache Kafka to HAQM MSK

August 30, 2023: HAQM Kinesis Data Analytics has been renamed to HAQM Managed Service for Apache Flink. Read the announcement in the AWS News Blog and learn more. This is a guest post by Delhivery. In this post, we describe the steps Delhivery took to migrate from self-managed Apache Kafka running on HAQM Elastic Compute […]

Streaming web content with a log-based architecture with HAQM MSK

Content, such as breaking news or sports scores, requires updates in near-real-time. To stay up to date, you may be constantly refreshing your browser or mobile app. Building APIs to deliver this content at speed and scale can be challenging. In this post, I present an alternative to an API-based approach. I outline the concept […]

How Goldman Sachs builds cross-account connectivity to their HAQM MSK clusters with AWS PrivateLink

August 2023: HAQM MSK now offers a managed feature called multi-VPC private connectivity to simplify connectivity of your Kafka clients to your brokers. Refer this blog to learn more. This guest post presents patterns for accessing an HAQM Managed Streaming for Apache Kafka cluster across your AWS account or HAQM Virtual Private Cloud (HAQM VPC) […]

Streaming ETL with Apache Flink and HAQM Kinesis Data Analytics

August 30, 2023: HAQM Kinesis Data Analytics has been renamed to HAQM Managed Service for Apache Flink. Read the announcement in the AWS News Blog and learn more. February 9, 2024: HAQM Kinesis Data Firehose has been renamed to HAQM Data Firehose. Read the AWS What’s New post to learn more. Most businesses generate data […]