AWS Big Data Blog
Category: Management Tools
Automate building data lakes using AWS Service Catalog
Today, organizations spend a considerable amount of time understanding business processes, profiling data, and analyzing data from a variety of sources. The result is highly structured and organized data used primarily for reporting purposes. These traditional systems extract data from transactional systems that consist of metrics and attributes that describe different aspects of the business. […]
Gain insights into your HAQM Kinesis Data Firehose delivery stream using HAQM CloudWatch
The volume of data being generated globally is growing at an ever-increasing pace. Data is generated to support an increasing number of use cases, such as IoT, advertisement, gaming, security monitoring, machine learning (ML), and more. The growth of these use cases drives both volume and velocity of streaming data and requires companies to capture, […]
Automate HAQM Redshift Cluster management operations using AWS CloudFormation
HAQM Redshift is a fast, petabyte-scale cloud data warehouse delivering the best price-performance. Tens of thousands of customers run business-critical workloads on HAQM Redshift. HAQM Redshift offers many features that enable you to build scalable, highly performant, cost-effective, and easy-to-manage workloads. For example, you can scale an HAQM Redshift cluster up or down based on […]
Auto scaling HAQM Kinesis Data Streams using HAQM CloudWatch and AWS Lambda
This post is co-written with Noah Mundahl, Director of Public Cloud Engineering at United Health Group. Update (12/1/2021): HAQM Kinesis Data Streams On-Demand mode is now the recommended way to natively auto scale your HAQM Kinesis Data Streams. In this post, we cover a solution to add auto scaling to HAQM Kinesis Data Streams. Whether […]
Increase HAQM Elasticsearch Service performance by upgrading to Graviton2
September 8, 2021: HAQM Elasticsearch Service has been renamed to HAQM OpenSearch Service. See details. HAQM OpenSearch Service supports multiple instance types based on your use case. In 2021, AWS announced general purpose (M6g), compute optimized (C6g), and memory optimized (R6g, R6gd) instance types for HAQM OpenSearch Service version 7.9 or later powered by AWS […]
Run usage analytics on HAQM QuickSight using AWS CloudTrail
HAQM QuickSight is a cloud-native BI service that allows end users to create and publish dashboards in minutes, without provisioning any servers or requiring complex licensing. You can view these dashboards on the QuickSight product console or embed them into applications and websites. After you deploy a dashboard, it’s important to assess how they and […]
Data monetization and customer experience optimization using telco data assets: Part 2
Part 1 of this series explains the importance of building and implementing a customer experience (CX) management and data monetization strategy for telecom service providers (TSPs), and the major challenges driving these initiatives. It also includes an AWS CloudFormation template to set up a demonstration of the solution using AWS services. It covers transforming and enriching […]
Securing access to EMR clusters using AWS Systems Manager
Organizations need to secure infrastructure when enabling access to engineers to build applications. Opening SSH inbound ports on instances to enable engineer access introduces the risk of a malicious entity running unauthorized commands. Using a Bastion host or jump server is a common approach used to allow engineer access to HAQM EMR cluster instances by […]
Accelerating HAQM Redshift federated query to HAQM Aurora MySQL with AWS CloudFormation
HAQM Redshift federated query allows you to combine data from one or more HAQM Relational Database Service (HAQM RDS) for MySQL and HAQM Aurora MySQL databases with data already in HAQM Redshift. You can also combine such data with data in an HAQM Simple Storage Service (HAQM S3) data lake. This post shows you how […]
Enhanced monitoring and automatic scaling for Apache Flink
August 30, 2023: HAQM Kinesis Data Analytics has been renamed to HAQM Managed Service for Apache Flink. Read the announcement in the AWS News Blog and learn more. Thousands of developers use Apache Flink to build streaming applications to transform and analyze data in real time. Apache Flink is an open-source framework and engine for […]