Airflow Architect
Abhinav SharmaUnited StatesMIDPosted 7d ago
Apache AirflowAWSBig DataData EngineeringCloud ArchitectureApache SparkPySparkHadoopKafkaPythonSQLShell ScriptingETLELTBatchNear Real-Time PipelinesData LakesLakehouseData WarehousingCI/CDPipeline AutomationMonitoringAlertingKubernetesDockerNoSQLData Modeling
About the Role
We are looking for an Airflow Architect to join our team. This role focuses on architecting enterprise-scale data pipelines and optimizing complex workflows.
Key Responsibilities
• Architect and implement enterprise-scale Apache Airflow orchestration frameworks
• Design, build, and optimize complex ETL/ELT pipelines
• Integrate Airflow with Spark, Kafka, Hadoop, Databricks, data lakes, and data warehouses
• Establish DAG development standards, monitoring, security, governance, and SLA management
• Lead AWS data platform implementations with a focus on performance and cost optimization
• Implement CI/CD and DevOps best practices for Airflow environments
• Mentor engineering teams and drive enterprise data architecture excellence
Requirements
• 10+ years of experience in Data Engineering and Big Data Platforms
• 3+ years of Enterprise Data Architecture experience
• Strong expertise in Apache Airflow and AWS data services
• Experience with Data Mesh, Data Fabric, and modern data platforms
• Core Java knowledge is a plus
• Experience leading architecture reviews and technical strategy discussions