AWS Big Data Blog

Category: Foundational (100)

Elevate your search and analytics skills with the new HAQM OpenSearch Service YouTube channel

We’re thrilled to announce the launch of the official HAQM OpenSearch Service YouTube channel—a comprehensive resource for anyone looking to master HAQM OpenSearch Service. Whether you’re just getting started with searches , vectors, analytics, or you’re looking to optimize large-scale implementations, our channel can be your go-to resource to help you unlock the full potential of OpenSearch Service.

A box indicating HAQM Redshift in the center of the image with boxes from right to left for HAQM RDS MySQL and PostgreSQL, HAQM Aurora MySQL and PostreSQL, HAQM EMR, HAQM Glue, HAQM S3 bucket, HAQM Managed Streaming for Apache Kafka and HAQM Kinesis. Each box has an arrow pointing to HAQM Redshift. Each arrow has the following labels: HAQM RDS & HAQM Aurora: zero-ETL and federated queries; AWS Glue and HAQM EMR: spark connector; HAQM S3 bucket: COPY command; HAQM Managed Streaming for Apache Kafka and HAQM Kinesis: redshift streaming. HAQM Data Firehose has an arrow pointing to HAQM S3 bucket indicating the data flow direction.

HAQM Redshift data ingestion options

HAQM Redshift, a warehousing service, offers a variety of options for ingesting data from diverse sources into its high-performance, scalable environment. Whether your data resides in operational databases, data lakes, on-premises systems, HAQM Elastic Compute Cloud (HAQM EC2), or other AWS services, HAQM Redshift provides multiple ingestion methods to meet your specific needs. The currently […]

HAQM OpenSearch Serverless cost-effective search capabilities, at any scale

We’re excited to announce the new lower entry cost for HAQM OpenSearch Serverless. With support for half (0.5) OpenSearch Compute Units (OCUs) for indexing and search workloads, the entry cost is cut in half. HAQM OpenSearch Serverless is a serverless deployment option for HAQM OpenSearch Service that you can use to run search and analytics workloads without the complexities […]

Architecture Diagram

Federate IAM-based single sign-on to HAQM Redshift role-based access control with Okta

HAQM Redshift accelerates your time to insights with fast, easy, and secure cloud data warehousing at scale. Tens of thousands of customers rely on HAQM Redshift to analyze exabytes of data and run complex analytical queries. You can use your preferred SQL clients to analyze your data in an HAQM Redshift data warehouse. Connect seamlessly by […]

Improve performance of workloads containing repetitive scan filters with multidimensional data layout sort keys in HAQM Redshift

HAQM Redshift, a widely used cloud data warehouse, has evolved significantly to meet the performance requirements of the most demanding workloads. This post covers one such new feature—the multidimensional data layout sort key. HAQM Redshift now improves your query performance by supporting multidimensional data layout sort keys, which is a new type of sort key […]

Build multi-layer maps in HAQM OpenSearch Service

With the release of HAQM OpenSearch Service 2.5, you can create maps with multiple layers to visualize your geographical data. You can build each layer from a different index pattern to separate data sources. Organizing the map in layers makes it more straightforward to visualize, view, and analyze geographical data. The layering also helps fetch […]

Using Experian identity resolution with AWS Clean Rooms to achieve higher audience activation match rates

This is a guest post co-written with Tyler Middleton, Experian Senior Partner Marketing Manager, and Jay Rakhe, Experian Group Product Manager. As the data privacy landscape continues to evolve, companies are increasingly seeking ways to collect and manage data while protecting privacy and intellectual property. First party data is more important than ever for companies […]

Deploy HAQM OpenSearch Serverless with Terraform

This post demonstrates how to use Terraform to create, deploy, and clean up OpenSearch Serverless infrastructure.. HAQM OpenSearch Serverless provides the search and analytical functionality of OpenSearch without the manual overhead of configuring, managing, and scaling OpenSearch clusters. It automatically scales the resources based on your workload, and you only pay for the resources consumed. Managing OpenSearch Serverless is simple, but with infrastructure as code (IaC) software like Terraform, you can simplify your resource management even more.

Monitoring HAQM OpenSearch Serverless using AWS User Notifications

HAQM OpenSearch Serverless is a serverless deployment option for HAQM OpenSearch Service that makes it simple for you to run search and analytics workloads without having to think about infrastructure management. The compute capacity used for data ingestion, and search and query in OpenSearch Serverless is measured in OpenSearch Compute Units (OCUs). Customers can configure […]

Try semantic search with the HAQM OpenSearch Service vector engine

HAQM OpenSearch Service has long supported both lexical and vector search, since the introduction of its kNN plugin in 2020. With recent developments in generative AI, including AWS’s launch of HAQM Bedrock earlier in 2023, you can now use HAQM Bedrock-hosted models in conjunction with the vector database capabilities of OpenSearch Service, allowing you to implement semantic search, retrieval augmented generation (RAG), recommendation engines, and rich media search based on high-quality vector search. The recent launch of the vector engine for HAQM OpenSearch Serverless makes it even easier to deploy such solutions.