Data Engineer/Architect - Apache Airflow - Astronomer

Project overview

Data Engineer/Architect - Apache Airflow - Astronomer

Project details

London - 3 days a week onsite (non-negotiable)

6 Months

£700-£800 per day Inside IR35

The Role

We're supporting a highly reputable client in the Financial Services industry who are seeking an experienced Data Engineer to support them through a period of significant change across their Technology Delivery function.

The team are in the early stages of adopting Apache Airflow and Astronomer as their core orchestration platform/tooling for their DBT projects. To move forward with the process and maximise the tooling, the team require an Engineer with expertise in this field to take technical ownership of establishing Astronomer as a production-ready, scalable, observable and cost-effective orchestration capability, working alongside our existing Data Engineering and platform teams. Responsibilities will include:

Essential Experience

Desirable Experience

Modis International Ltd acts as an employment agency for permanent recruitment and an employment business for the supply of temporary workers in the UK. Modis Europe Ltd provide a variety of international solutions that connect clients to the best talent in the world. For all positions based in Switzerland, Modis Europe Ltd works with its licensed Swiss partner Accurity GmbH to ensure that candidate applications are handled in accordance with Swiss law.

Both Modis International Ltd and Modis Europe Ltd are Equal Opportunities Employers.

By applying for this role your details will be submitted to Modis International Ltd and/ or Modis Europe Ltd. Our Candidate Privacy Information Statement which explains how we will use your information is available on the Modis website.

• Defining reusable Architectural and Orchestration patterns for DBT and Snowflake Workloads
• Establishing appropriate security, secrets, RBAC and environment controls
• Implement monitoring, alerting and observability covering platform and workload health.
• Defining reliable integration patterns between Airflow and our client's wider enterprise scheduling environment (Autosys etc)
• Provide upskilling and documentation to the wider team to aid with the option of Apache Airflow / Astronomer Cosmos
• Highly experienced Data Engineer/Architect with a broad range of enterprise environment experience
• Strong background with DBT, Snowflake and Python
• Previous experience working with Apache Airflow and Astronomer, with a strong preference towards Astronomer Cosmos experience
• Experience on an Airflow/Astronomer implementation would be preferred
• Excellent communication and stakeholder engagement skills
• Experience supporting client's in the Financial Services sector
• Enterprise scheduler tooling experience, with a preference towards AutoSys
• CI/CD, testing and deployment experience
• Azure DevOps and/or Github Actions experience
• Start: flexible
• Location: London, South East England, GB, GB
• Rate: 88-100 EUR/h