Tag: airflow

Apache Airflow is an open-source distributed workflow management platform for authoring, scheduling, and monitoring multi-stage workflows. It is designed to be extensible, and it’s compatible with several services like HAQM Elastic Kubernetes Service (HAQM EKS), HAQM Elastic Container Service (HAQM ECS), and HAQM EC2. Many AWS customers choose to run Airflow on containerized environments with […]

Apache Airflow is an open-source distributed workflow management platform that allows you to schedule, orchestrate, and monitor workflows. Airflow helps you automate and orchestrate complex data pipelines that can be multistep with inter-dependencies. This post presents a reference architecture where Airflow runs entirely on AWS Fargate with HAQM Elastic Container Service (ECS) as the orchestrator, […]