AWS Big Data Blog
Category: HAQM Managed Service for Apache Flink
Enhanced monitoring and automatic scaling for Apache Flink
August 30, 2023: HAQM Kinesis Data Analytics has been renamed to HAQM Managed Service for Apache Flink. Read the announcement in the AWS News Blog and learn more. Thousands of developers use Apache Flink to build streaming applications to transform and analyze data in real time. Apache Flink is an open-source framework and engine for […]
Stream, transform, and analyze XML data in real time with HAQM Kinesis, AWS Lambda, and HAQM Redshift
August 30, 2023: HAQM Kinesis Data Analytics has been renamed to HAQM Managed Service for Apache Flink. Read the announcement in the AWS News Blog and learn more. February 9, 2024: HAQM Kinesis Data Firehose has been renamed to HAQM Data Firehose. Read the AWS What’s New post to learn more. When we look at […]
Best practices from Delhivery on migrating from Apache Kafka to HAQM MSK
August 30, 2023: HAQM Kinesis Data Analytics has been renamed to HAQM Managed Service for Apache Flink. Read the announcement in the AWS News Blog and learn more. This is a guest post by Delhivery. In this post, we describe the steps Delhivery took to migrate from self-managed Apache Kafka running on HAQM Elastic Compute […]
Streaming ETL with Apache Flink and HAQM Kinesis Data Analytics
August 30, 2023: HAQM Kinesis Data Analytics has been renamed to HAQM Managed Service for Apache Flink. Read the announcement in the AWS News Blog and learn more. February 9, 2024: HAQM Kinesis Data Firehose has been renamed to HAQM Data Firehose. Read the AWS What’s New post to learn more. Most businesses generate data […]
Build and run streaming applications with Apache Flink and HAQM Kinesis Data Analytics for Java Applications
In this post, we discuss how you can use Apache Flink and HAQM Kinesis Data Analytics for Java Applications to address these challenges. We explore how to build a reliable, scalable, and highly available streaming architecture based on managed services that substantially reduce the operational overhead compared to a self-managed environment.
Create real-time clickstream sessions and run analytics with HAQM Kinesis Data Analytics, AWS Glue, and HAQM Athena
April 2024: The content of this post is no longer relevant and deprecated. August 30, 2023: HAQM Kinesis Data Analytics has been renamed to HAQM Managed Service for Apache Flink. Read the announcement in the AWS News Blog and learn more. Clickstream events are small pieces of data that are generated continuously with high speed […]
Analyze and visualize your VPC network traffic using HAQM Kinesis and HAQM Athena
In this blog post, we describe the complete solution for collecting, analyzing, and visualizing VPC flow log data. In addition, we created a single AWS CloudFormation template that lets you efficiently deploy this solution into your own account.
Real-time bushfire alerting with Complex Event Processing in Apache Flink on HAQM EMR and IoT sensor network
In this blog post, we discuss how to build a real-time IoT stream processing, visualization, and alerting pipeline using various AWS services. We took advantage of the Complex Event Processing feature provided by Apache Flink to detect patterns within a network from the incoming events.
Build a blockchain analytic solution with AWS Lambda, HAQM Kinesis, and HAQM Athena
In this post, we’ll show you how to deploy an Ethereum blockchain using the AWS Blockchain Templates, deploy a smart contract, and build a serverless analytics pipeline for that contract based around AWS Lambda, HAQM Kinesis, and HAQM Athena.
Optimize Delivery of Trending, Personalized News Using HAQM Kinesis and Related Services
Gunosy aims to provide people with the content they want without the stress of dealing with a large influx of information. We analyze user attributes, such as gender and age, and past activity logs like click-through rate (CTR). We combine this information with article attributes to provide trending, personalized news articles to users. In this post, I show you how to process user activity logs in real time using HAQM Kinesis Data Firehose, HAQM Kinesis Data Analytics, and related AWS services.