AWS Big Data Blog

Category: AWS Identity and Access Management (IAM)

Accelerate your analytics with HAQM S3 Tables and HAQM SageMaker Lakehouse

HAQM SageMaker Lakehouse is a unified, open, and secure data lakehouse that now seamlessly integrates with HAQM S3 Tables, the first cloud object store with built-in Apache Iceberg support. In this post, we guide you how to use various analytics services using the integration of SageMaker Lakehouse with S3 Tables.

Federate to HAQM Redshift Query Editor v2 with Microsoft Entra ID

In this post, we explore the process of federating into AWS using Microsoft Entra ID and AWS Identity and Access Management (IAM), and how to restrict access to datasets based on permissions linked to AD groups. We guide you through the setup process, and demonstrate how to seamlessly connect to the Redshift Query Editor while making sure data access permissions are accurately enforced based on your Microsoft Entra ID groups.

Take manual snapshots and restore in a different domain spanning across various Regions and accounts in HAQM OpenSearch Service

This post provides a detailed walkthrough about how to efficiently capture and manage manual snapshots in OpenSearch Service. It covers the essential steps for taking snapshots of your data, implementing safe transfer across different AWS Regions and accounts, and restoring them in a new domain. This guide is designed to help you maintain data integrity and continuity while navigating complex multi-Region and multi-account environments in OpenSearch Service.

Integrate Tableau and Okta with HAQM Redshift using AWS IAM Identity Center

This blog post is co-written with Sid Wray and Jake Koskela from Salesforce, and Adiascar Cisneros from Tableau.  HAQM Redshift is a fast, scalable cloud data warehouse built to serve workloads at any scale. With HAQM Redshift as your data warehouse, you can run complex queries using sophisticated query optimization to quickly deliver results to […]

HAQM MSK IAM authentication now supports all programming languages

The AWS Identity and Access Management (IAM) authentication feature in HAQM Managed Streaming for Apache Kafka (HAQM MSK) now supports all programming languages. Administrators can simplify and standardize access control to Kafka resources using IAM. This support is based on SASL/OUATHBEARER, an open standard for authorization and authentication. Both HAQM MSK provisioned and serverless cluster […]

Build streaming data pipelines with HAQM MSK Serverless and IAM authentication

HAQM’s serverless Apache Kafka offering, HAQM Managed Streaming for Apache Kafka (HAQM MSK) Serverless, is attracting a lot of interest. It’s appreciated for its user-friendly approach, ability to scale automatically, and cost-saving benefits over other Kafka solutions. However, a hurdle encountered by many users is the requirement of MSK Serverless to use AWS Identity and Access Management (IAM) access control. At the time of writing, the HAQM MSK library for IAM is exclusive to Kafka libraries in Java, creating a challenge for users of other programming languages. In this post, we aim to address this issue and present how you can use HAQM API Gateway and AWS Lambda to navigate around this obstacle.

Multi-tenancy Apache Kafka clusters in HAQM MSK with IAM access control and Kafka Quotas – Part 1

With HAQM Managed Streaming for Apache Kafka (HAQM MSK), you can build and run applications that use Apache Kafka to process streaming data. To process streaming data, organizations either use multiple Kafka clusters based on their application groupings, usage scenarios, compliance requirements, and other factors, or a dedicated Kafka cluster for the entire organization. It […]

Multi-tenancy Apache Kafka clusters in HAQM MSK with IAM access control and Kafka quotas – Part 2

Kafka quotas are integral to multi-tenant Kafka clusters. They prevent Kafka cluster performance from being negatively affected by poorly behaved applications overconsuming cluster resources. Furthermore, they enable the central streaming data platform to be operated as a multi-tenant platform and used by downstream and upstream applications across multiple business lines. Kafka supports two types of quotas: […]

Federate HAQM QuickSight access with open-source identity provider Keycloak

HAQM QuickSight is a scalable, serverless, embeddable, machine learning (ML) powered business intelligence (BI) service built for the cloud that supports identity federation in both Standard and Enterprise editions. Organizations are working toward centralizing their identity and access strategy across all their applications, including on-premises and third-party. Many organizations use Keycloak as their identity provider […]

High-level data platform expected behavior

How Novo Nordisk built distributed data governance and control at scale

This is a guest post co-written with Jonatan Selsing and Moses Arthur from Novo Nordisk. This is the second post of a three-part series detailing how Novo Nordisk, a large pharmaceutical enterprise, partnered with AWS Professional Services to build a scalable and secure data and analytics platform. The first post of this series describes the […]