AWS Big Data Blog

Category: HAQM Kinesis

Ingest VPC flow logs into Splunk using HAQM Kinesis Data Firehose

February 9, 2024: HAQM Kinesis Data Firehose has been renamed to HAQM Data Firehose. Read the AWS What’s New post to learn more. December 2023: This post was reviewed and updated to remove the dependency on the AWS Lambda function according to the latest version in Splunk AWS Add-on (7.3.0). In September 2017, during the […]

Common streaming data enrichment patterns in HAQM Kinesis Data Analytics for Apache Flink

August 30, 2023: HAQM Kinesis Data Analytics has been renamed to HAQM Managed Service for Apache Flink. Read the announcement in the AWS News Blog and learn more. Stream data processing allows you to act on data in real time. Real-time data analytics can help you have on-time and optimized responses while improving overall customer […]

How NerdWallet uses AWS and Apache Hudi to build a serverless, real-time analytics platform

This is a guest post by Kevin Chun, Staff Software Engineer in Core Engineering at NerdWallet. NerdWallet’s mission is to provide clarity for all of life’s financial decisions. This covers a diverse set of topics: from choosing the right credit card, to managing your spending, to finding the best personal loan, to refinancing your mortgage. […]

Best practices to optimize cost and performance for AWS Glue streaming ETL jobs

AWS Glue streaming extract, transform, and load (ETL) jobs allow you to process and enrich vast amounts of incoming data from systems such as HAQM Kinesis Data Streams, HAQM Managed Streaming for Apache Kafka (HAQM MSK), or any other Apache Kafka cluster. It uses the Spark Structured Streaming framework to perform data processing in near-real […]

Analyze logs with Dynatrace Davis AI Engine using HAQM Kinesis Data Firehose HTTP endpoint delivery

February 9, 2024: HAQM Kinesis Data Firehose has been renamed to HAQM Data Firehose. Read the AWS What’s New post to learn more. This blog post is co-authored with Erick Leon, Sr. Technical Alliance Manager from Dynatrace. HAQM Kinesis Data Firehose is the easiest way to reliably load streaming data into data lakes, data stores, and […]

BDB-2063-kda-keyspaces-architecture

Sink HAQM Kinesis Data Analytics Apache Flink output to HAQM Keyspaces using Apache Cassandra Connector

August 30, 2023: HAQM Kinesis Data Analytics has been renamed to HAQM Managed Service for Apache Flink. Read the announcement in the AWS News Blog and learn more. HAQM Keyspaces (for Apache Cassandra) is a scalable, highly available, and managed Apache Cassandra–compatible database service. With HAQM Keyspaces you don’t have to provision, patch, or manage […]

Stream change data to HAQM Kinesis Data Streams with AWS DMS

August 30, 2023: HAQM Kinesis Data Analytics has been renamed to HAQM Managed Service for Apache Flink. Read the announcement in the AWS News Blog and learn more. In this post, we discuss how to use AWS Database Migration Service (AWS DMS) native change data capture (CDC) capabilities to stream changes into HAQM Kinesis Data […]

A serverless operational data lake for retail with AWS Glue, HAQM Kinesis Data Streams, HAQM DynamoDB, and HAQM QuickSight

Do you want to reduce stockouts at stores? Do you want to improve order delivery timelines? Do you want to provide your customers with accurate product availability, down to the millisecond? A retail operational data lake can help you transform the customer experience by providing deeper insights into a variety of operational aspects of your […]

Build a big data Lambda architecture for batch and real-time analytics using HAQM Redshift

February 9, 2024: HAQM Kinesis Data Firehose has been renamed to HAQM Data Firehose. Read the AWS What’s New post to learn more. With real-time information about customers, products, and applications in hand, organizations can take action as events happen in their business application. For example, you can prevent financial fraud, deliver personalized offers, and […]

Use HAQM Kinesis Data Firehose to extract data insights with Coralogix

February 9, 2024: HAQM Kinesis Data Firehose has been renamed to HAQM Data Firehose. Read the AWS What’s New post to learn more. This is a guest blog post co-written by Tal Knopf at Coralogix. Digital data is expanding exponentially, and the existing limitations to store and analyze it are constantly being challenged and overcome. […]