HAQM Redshift announces support for Confluent Cloud and Apache Kafka
HAQM Redshift now supports streaming ingestion from Confluent Managed Cloud and self-managed Apache Kafka clusters on HAQM EC2 instances, expanding its capabilities beyond HAQM Kinesis Data Streams (KDS) and HAQM Managed Streaming for Apache Kafka (MSK).
With this update, customers can ingest data from a wider range of streaming sources directly into their HAQM Redshift data warehouses. HAQM Redshift introduces mTLS (mutual Transport Layer Security) as the authentication protocol for secure communication between HAQM Redshift and the newly supported Kafka streaming sources. This ensures that data ingestion from these new sources maintains the high security standards expected in enterprise data workflows. Additionally, a new SQL identifier 'KAFKA' has been introduced to simplify the identification of these newly supported Kafka sources in HAQM Redshift External Schema definitions.
You can start using this expanded streaming ingestion capability immediately, to build more comprehensive and flexible data pipelines that ingest data from various Kafka sources — those offered by AWS (HAQM MSK), those available from partners (Confluent Cloud) or those that are self-managed (Apache Kafka) on HAQM EC2.
To learn more and get started with streaming data into HAQM Redshift from any Kafka source, refer to the HAQM Redshift streaming documentation.