AWS Big Data Blog
Category: HAQM Kinesis
Build a data lake using HAQM Kinesis Data Streams for HAQM DynamoDB and Apache Hudi
August 30, 2023: HAQM Kinesis Data Analytics has been renamed to HAQM Managed Service for Apache Flink. Read the announcement in the AWS News Blog and learn more. July 2023: This post was reviewed for accuracy. HAQM DynamoDB helps you capture high-velocity data such as clickstream data to form customized user profiles and online order […]
Retaining data streams up to one year with HAQM Kinesis Data Streams
August 30, 2023: HAQM Kinesis Data Analytics has been renamed to HAQM Managed Service for Apache Flink. Read the announcement in the AWS News Blog and learn more. Streaming data is used extensively for use cases like sharing data between applications, streaming ETL (extract, transform, and load), real-time analytics, processing data from internet of things […]
How Baqend built a real-time web analytics platform using HAQM Kinesis Data Analytics for Apache Flink
August 30, 2023: HAQM Kinesis Data Analytics has been renamed to HAQM Managed Service for Apache Flink. Read the announcement in the AWS News Blog and learn more. This is a customer post written by the engineers from German startup Baqend and the AWS EMEA Prototyping Labs team. Baqend is one of the fastest-growing software […]
Validate, evolve, and control schemas in HAQM MSK and HAQM Kinesis Data Streams with AWS Glue Schema Registry
August 30, 2023: HAQM Kinesis Data Analytics has been renamed to HAQM Managed Service for Apache Flink. Read the announcement in the AWS News Blog and learn more. Data streaming technologies like Apache Kafka and HAQM Kinesis Data Streams capture and distribute data generated by thousands or millions of applications, websites, or machines. These technologies […]
Building a real-time notification system with HAQM Kinesis Data Streams for HAQM DynamoDB and HAQM Kinesis Data Analytics for Apache Flink
August 30, 2023: HAQM Kinesis Data Analytics has been renamed to HAQM Managed Service for Apache Flink. Read the announcement in the AWS News Blog and learn more. HAQM DynamoDB helps you capture high-velocity data such as clickstream data to form customized user profiles and Internet of Things (IoT) data so that you can develop […]
Building an ad-to-order conversion engine with HAQM Kinesis, AWS Glue, and HAQM QuickSight
August 30, 2023: HAQM Kinesis Data Analytics has been renamed to HAQM Managed Service for Apache Flink. Read the announcement in the AWS News Blog and learn more. Businesses in ecommerce have the challenge of measuring their ad-to-order conversion ratio for ads or promotional campaigns displayed on a webpage. Tracking the number of users that […]
Building a scalable streaming data processor with HAQM Kinesis Data Streams on AWS Fargate
Data is ubiquitous in businesses today, and the volume and speed of incoming data are constantly increasing. To derive insights from data, it’s essential to deliver it to a data lake or a data store and analyze it. Real-time or near-real-time data delivery can be cost prohibitive, therefore an efficient architecture is key for processing, […]
Best practices for consuming HAQM Kinesis Data Streams using AWS Lambda
November 2024: This post was reviewed and updated for accuracy. Many organizations are processing and analyzing clickstream data in real time from customer-facing applications to look for new business opportunities and identify security incidents in real time. A common practice is to consolidate and enrich logs from applications and servers in real time to proactively […]
Detect change points in your event data stream using HAQM Kinesis Data Streams, HAQM DynamoDB and AWS Lambda
The success of many modern streaming applications depends on the ability to sequentially detect each change as soon as possible after it occurs, while continuing to monitor the data stream as it evolves. Applications of change point detection range across genomics, marketing, and finance, to name a few. In genomics, change point detection can help […]
Migrating from Vertica to HAQM Redshift
HAQM Redshift powers analytical workloads for Fortune 500 companies, startups, and everything in between. With HAQM Redshift, you can query petabytes of structured and semi-structured data across your data warehouse, operational database, and your data lake using standard SQL. When you use Vertica, you have to install and upgrade Vertica database software and manage the […]