This website uses cookies to ensure you get the best experience. Learn more
Synechron

Airflow Developer

Job Description

We are looking for an experienced Airflow Developer to work for our client in the New York or New Jersey office.

Responsibilities

  • Develop guidelines for Airflow cluster and DAG's. Performance tuning of the DAG's and task implementation.
  • Develop DAG - data pipeline to on-board and change management of datasets.
  • Implement CI/CD for Airflow DAG deployment and test automation frameworks.
  • Build DAG to orchestrate augmented pipeline implementing models in pipeline, running reconciliation, DQ, etc.  
  • Orchestrating the Airflow / workflow in hybrid cloud environment is big plus
  • Working in a hybrid cloud environment.
  • Develop and contribute across the stack (Javascript/React, Java back end).
  • Developing hooks, templates, Alerts and scheduling for pipelines in Airflow.

Requirements

  • Minimum 2-3 years of experience on Apache Airflow in recent project.
  • Python & Pandas, with demonstrated experience in ETL.
  • Ability to maintain clean and secure data environments.
  • Experience in setting and Schedulers: celery, Dask, Mesos.
  • Well-versed with messaging queue : RabbitMQ or Redis.
  • Understanding of airflow rest services and integration of airflow of platform eco-system.
  • Experience working in an Agile or iterative approach to delivery preferred.
  • Experience with Apache Airflow, including installing, configuring and monitoring Airflow cluster.

 

Apply Now

Share this