Databricks Data Engineer

Project overview

For our direct end client, we are looking for a Senior Databricks Data Engineer (m/f/d) starting immediately.

Responsibilities

• Implementation and further development of the Azure/Databricks target architecture (Lakehouse)
• Migration of existing ETL processes (Informatica/Oracle) into performant Databricks pipelines
• Development of data pipelines with Python, PySpark, and SQL
• Setup and maintenance of Delta Lake structures as well as pipelines with DLT/SDP
• Implementation of data models and historization, including according to SCD2
• Ensuring data quality and governance, including with Unity Catalog and authorization models
• Performance optimization when processing large amounts of data
• Support for tests, parallel operation, and cutover
• Coordination with business departments and other development teams as well as documentation of work results

Requirements

• Sound practical experience in implementing Lakehouse architectures as well as in setting up and operating Databricks
• Deep knowledge of Delta Lake, Unity Catalog, DLT/SDP, authorization models, and governance mechanisms
• Sound experience in Python, PySpark, and SQL as well as in performance optimization of large data volumes
• Confident use of AI-based tools for increasing efficiency in development
• Sound experience in the classic BI/DWH environment, i.e., in data storage, data modeling, data transport, and historization according to SCD2
• Experience as a developer in a Scrum team
• Ideally, experience in replacing classic ETL routes (e.g., Informatica) and migrating them to Databricks
• Excellent German language skills, both written and spoken, as well as confident, clear communication with business departments and development teams

Project details

Task Description

As part of the role as Databricks Data Engineer, the existing Oracle/Informatica DWH will be migrated to a modern Azure Lakehouse architecture based on Databricks. In close cooperation with the business departments and other Data Engineers, robust and scalable data solutions will be created. Work will be done iteratively and customer-oriented in an established Scrum setup.

• Start: Immediately, duration until December 31, 2026 (extension planned, long-term collaboration desired)
• Location: Cologne, approx. 40% on-site (2 days per week) and 60% remote
• Residence: Maximum 100 km around Cologne
• Utilization: Full-time
• Start: 2026-12-01
• Duration: 12 Monate
• Location: Köln, DE
• Languages: German