Senior Data Engineer (Azure Databricks)

Project overview

For our client, an insurance company in the greater Frankfurt am Main area, we are looking for two experienced Data Engineers as part of the development of a new dispositional data platform (Lakehouse) based on Microsoft Azure and Databricks. You will build the data pipelines from the source systems to the core model, taking technical leadership in the implementation. You will work in tandem with client employees and actively share your Databricks know-how.

Responsibilities

• Connecting source systems and external service providers (Delta deliveries, files, APIs) as well as a CRM system, including automated loading processes
• Development of ETL/ELT pipelines with Delta processing, error handling, and traceable restart procedures
• Implementation of duplicate cleansing and merging of customer and intermediary data
• Implementation of the core data model with historization along the Medallion architecture (Databricks, PySpark, SQL)
• Implementation of data quality rules, check routines, monitoring, and alerting
• Provision of exports and feedback to operational systems (e.g., campaign selections for CRM)
• Contribution to development pipelines and CI/CD, roles and permissions, Data Catalog, and Lineage
• Documentation, runbooks, and knowledge transfer in tandem with the client team

Requirements

General Framework Conditions

• Several years of experience as a Data Engineer, ideally in building data warehouse or lakehouse solutions
• Very good knowledge of Databricks, PySpark/Spark, SQL, and Delta Lake
• Practical experience with Databricks Workflows or Azure Data Factory and other Azure data services
• Experience in data modeling and historization (Data Vault and/or dimensional modeling)
• Knowledge of Git, CI/CD, and automated testing of data pipelines
• Experience with AI-supported development (e.g., coding assistants, agentic workflows) is an advantage
• Experience with source systems and data in the insurance environment is an advantage
• Enjoyment of knowledge transfer and collaboration in a mixed team
• Databricks or Azure Data Engineer certification is an advantage
• Fluent German and good English skills
• Workload: 100%
• Start: Beginning of November 2026
• Duration: until 03/31/2027 with an option for extension (follow-up phases planned until the end of 2028)
• Work location: Greater Frankfurt am Main area (Germany)
• Onsite/Remote: High onsite component in the first 2–3 months (3–4 days per week), then approx. 2 days onsite, rest remote
• Client meeting: Availability for interviews with the client at the end of October 2026

Project details

• Start: 2026-11
• Duration: 6 Monate
• Remote: hybrid
• Location: Frankfurt, Deutschland, DE
• Languages: German