Category: HAQM Redshift

The materialized views feature in HAQM Redshift is now generally available and has been benefiting customers and partners in preview since December 2019. One customer, AXS, is a leading ticketing, data, and marketing solutions provider for live entertainment venues in the US, UK, Europe, and Japan. Etleap, an HAQM Redshift partner, is an extract, transform, […]

Today’s analytics workloads typically require a data warehouse to be available 24 hours a day, 7 days a week. However, there may be times when you need an HAQM Redshift cluster for a short duration of time at frequent (or infrequent) intervals. For example, you may run a periodic ETL job or use a cluster […]

The AWS Cost & Usage Report (CUR) tracks your AWS usage and provides estimated charges associated with that usage. You can configure this report to present the data at hourly or daily intervals, and it is updated at least one time per day until it is finalized at the end of the billing period. The […]

The annual AWS re:Invent learning conference is an exciting time full of new product and program launches. At the first re:Invent conference in 2012, AWS announced HAQM Redshift. Since then, tens of thousands of customers have started using HAQM Redshift as their cloud data warehouse. In 2019, AWS shared several significant launches and dozens of […]

This is a guest post from ZS. In their own words, “ZS is a professional services firm that works closely with companies to help develop and deliver products and solutions that drive customer value and company results. ZS engagements involve a blend of technology, consulting, analytics, and operations, and are targeted toward improving the commercial […]

Redshift Spectrum is a feature of HAQM Redshift that allows you to query data stored on HAQM S3 directly and supports nested data types. This post discusses which use cases can benefit from nested data types, how to use HAQM Redshift Spectrum with nested data types to achieve excellent performance and storage efficiency, and some […]

New: Read HAQM Redshift continues its price-performance leadership to learn what analytic workload trends we’re seeing from HAQM Redshift customers, new capabilities we have launched to improve Redshift’s price-performance, and the results from the latest benchmarks. Part 1 of this multi-post series, ETL and ELT design patterns for modern data architecture using HAQM Redshift: Part 1, […]

New: Read HAQM Redshift continues its price-performance leadership to learn what analytic workload trends we’re seeing from HAQM Redshift customers, new capabilities we have launched to improve Redshift’s price-performance, and the results from the latest benchmarks. Part 1 of this multi-post series discusses design best practices for building scalable ETL (extract, transform, load) and ELT (extract, […]

HAQM Redshift is the most popular and the fastest cloud data warehouse. It includes a console for administrators to create, configure, and manage HAQM Redshift clusters. The new HAQM Redshift console modernizes the user interface and adds several features to improve managing your clusters and workloads running on clusters. The new HAQM Redshift console provides visibility to the health and performance of clusters from a unified dashboard, simplified management of clusters by streamlining several screens and flows, improved mean-time-to-diagnose query performance issues by adding capabilities to monitor user queries and correlate with cluster performance metrics, as well as the ability for non-admin users to use Query Editor.

In this post, I show how to use AWS Step Functions and AWS Glue Python Shell to orchestrate tasks for those HAQM Redshift-based ETL workflows in a completely serverless fashion. AWS Glue Python Shell is a Python runtime environment for running small to medium-sized ETL tasks, such as submitting SQL queries and waiting for a response. Step Functions lets you coordinate multiple AWS services into workflows so you can easily run and monitor a series of ETL tasks. Both AWS Glue Python Shell and Step Functions are serverless, allowing you to automatically run and scale them in response to events you define, rather than requiring you to provision, scale, and manage servers.