AWS Big Data Blog

In this post, we discuss how HAQM Redshift spatial index functions such as Hexagonal hierarchical geospatial indexing system (or H3) can be used to represent spatial data using H3 indexing for fast spatial lookups at scale. Navigating the vast landscape of data-driven insights has always been an exciting endeavor. As technology continues to evolve, one specific facet of this journey is reaching unprecedented proportions: geospatial data.

As data analytics use cases grow, factors of scalability and concurrency become crucial for businesses. Your analytic solution architecture should be able to handle large data volumes at high concurrency and without compromising speed, thereby delivering a scalable high-performance analytics environment. HAQM Redshift Serverless provides a fully managed, petabyte-scale, auto scaling cloud data warehouse to […]

In this post, we show you how to use the new views feature the AWS Glue Data Catalog. SQL views are a powerful object used across relational databases. You can use views to decrease the time to insights of data by tailoring the data that is queried. Additionally, you can use the power of SQL […]

Data governance is a key enabler for teams adopting a data-driven culture and operational model to drive innovation with data. HAQM DataZone is a fully managed data management service that makes it faster and easier for customers to catalog, discover, share, and govern data stored across HAQM Web Services (AWS), on premises, and on third-party […]

In today’s data-driven landscape, the efficiency and accessibility of querying tools play a crucial role in driving businesses forward. HAQM Redshift recently announced integration with Visual Studio Code (), an action that transforms the way data practitioners engage with HAQM Redshift and reshapes your interactions and practices in data management. This innovation not only unlocks […]

In today’s data-driven landscape, managing and analyzing vast amounts of data, especially logs, is crucial for organizations to derive insights and make informed decisions. However, handling this data efficiently presents a significant challenge, prompting organizations to seek scalable solutions without the complexity of infrastructure management. HAQM OpenSearch Serverless lets you run OpenSearch in the AWS […]

October 2024: This post was reviewed and updated for accuracy. AWS Glue is a fully managed, serverless data integration service provided by HAQM Web Services (AWS) that uses Apache Spark as one of its backend processing engines (as of this writing, you can use Python Shell or Spark). Data skew occurs when the data being […]

This is a guest post co-authored with Kanehito Miyake, Engineer at Fujitsu Japan.  Fujitsu Limited was established in Japan in 1935. Currently, we have approximately 120,000 employees worldwide (as of March 2023), including group companies. We develop business in various regions around the world, starting with Japan, and provide digital services globally. To provide a […]

Today, we’re excited to announce general availability of HAQM Q data integration in AWS Glue. HAQM Q data integration, a new generative AI-powered capability of HAQM Q Developer, enables you to build data integration pipelines using natural language. This reduces the time and effort you need to learn, build, and run data integration jobs using […]

The construction of big data applications based on open source software has become increasingly uncomplicated since the advent of projects like Data on EKS, an open source project from AWS to provide blueprints for building data and machine learning (ML) applications on HAQM Elastic Kubernetes Service (HAQM EKS). In the realm of big data, securing […]