AWS Open Source Blog

Category: HAQM Managed Workflows for Apache Airflow (HAQM MWAA)

phonlamaiphoto - stock.adobe.com cloud computing technology

Deploying to HAQM Managed Workflows for Apache Airflow with CI/CD tools

Apache Airflow‘s active open source community, familiar Python development as directed acyclic graph (DAG) workflows, and extensive library of pre-built integrations have helped it become a leading tool for data scientists and engineers for creating data pipelines. HAQM Managed Workflows for Apache Airflow (HAQM MWAA) is a fully managed service that makes running open source […]

solution architecture described in blog post

HAQM MWAA with AWS CodeArtifact for Python dependencies

This post was written by Dzenan Softic and Sam Dengler. Many organizations rely on Apache Airflow, an open source project, to orchestrate their data pipelines. In 2020, HAQM Web Services (AWS) released HAQM Managed Workflows for Apache Airflow (HAQM MWAA), which lets engineers focus on business solutions rather than on running and maintaining infrastructure for […]

Automating a DAG deployment with HAQM Managed Workflows for Apache Airflow

Many developers and data engineers use Managed Workflows for Apache Airflow (HAQM MWAA), a managed service for open source Apache Airflow, to programmatically author, schedule, and monitor workflows. With HAQM MWAA, you can focus on business logic and build workflows without worrying about the management responsibilities such as setup, patching, upgrades, security, scaling, and so […]

Tierney - stock.adobe.com Cloud computing with abstract high speed technology POV motion blur

Migrating from self-managed Apache Airflow to HAQM Managed Workflows for Apache Airflow (MWAA)

This post was written by Tomas Christ, Solution Architect at eprimo GmbH. eprimo GmbH is a wholly owned subsidiary of E.ON SE, situated near Frankfurt, Germany. It represents the largest purely green-energy supplier in Germany with some 1.7 million customers. Currently, eprimo has a staff of approximately 160 people. We have been using HAQM Web […]

Chepko Danil – stock.adobe.com

Move your Apache Airflow connections and variables to AWS Secrets Manager

Data scientists and engineers have made Apache Airflow a leading open source tool to create data pipelines due to its active open source community, familiar Python development as directed acyclic graph (DAG) workflows, and extensive library of prebuilt integrations. However, managing the connections and variables that these pipelines depend on can be a challenge, especially […]

peshkov – stock.adobe.com AdobeStock_303323028_news_1000x500

HAQM Managed Workflows for Apache Airflow unaffected by Airflow 1.10.12 vulnerability

HAQM Managed Workflows for Apache Airflow (MWAA) is not affected by the recently announced vulnerability in Apache Airflow 1.10.12. The default airflow.cfg file uses a temporary key that is the same for all installations. In Airflow 1.10.12 and earlier, there was no restriction in using that temporary key on the Airflow web server, meaning that […]