(Senior) Databricks Engineer – Freelance & Remote

Project overview

Project Context & Objective

For strategic transformation projects and the expansion of modern, scalable data platforms, we are looking for an experienced (Senior) Databricks Engineer (m/f/d). In this role, you will take on technical responsibility from the conception to the productive rollout of Data & AI solutions based on state-of-the-art Lakehouse architectures.

Responsibilities

Skills

Must-Have Skills

• Platform Conception & Architecture: Conception, setup, and further development of modern Lakehouse architectures and scalable data platforms on Databricks.
• Pipeline Development: Development of high-performance ETL/ELT pipelines (batch and streaming) using Python, SQL, PySpark, and Apache Spark.
• Databricks Ecosystem & Tooling: Implementation of modern components such as Unity Catalog, Delta Lake, and Delta Live Tables / Lakeflow; integration into existing cloud and analytics landscapes.
• Data Governance & Security: Establishment of robust governance structures, including data classification, lineage, data quality frameworks, and access controls.
• Automation & Performance: Setup of automated deployments (CI/CD, Infrastructure as Code) and continuous performance and cost optimization of existing Databricks workloads.
• Technical Consulting & Knowledge Transfer: Technical sparring partner role for stakeholders, coaching internal data engineers, and participation in Proof of Concepts (PoCs).
• In-depth Databricks Expertise: Several years of project experience in the design and operation of productive Databricks solutions.
• Data Engineering & Programming Languages: Excellent knowledge of Python, SQL, and sound practical experience with PySpark and distributed data processing (Apache Spark).
• Architecture & Data Modeling: Demonstrable experience with Delta Lake, Lakehouse paradigms, and relational/dimensional data modeling.
• Data Governance & Security: Hands-on experience with Databricks Unity Catalog (role/permission management, lineage, data quality).
• Cloud Infrastructure: Solid project experience in at least one major cloud environment (Microsoft Azure, AWS, or GCP).
• Software Engineering Best Practices: Proficient use of Git, CI/CD pipelines, testing, and IaC (e.g., Terraform).
• Communication & Work Style: Independent, structured way of working, strong architectural understanding, and fluent German and good English skills.

Nice to have

• Databricks certifications (e.g., Databricks Certified Data Engineer Professional).
• Experience with Delta Live Tables (DLT) or Databricks Lakeflow.
• Experience in the technical conception of PoCs and customer coaching.

Project details

• Start: 02.11.2026
• Duration: unbefristet
• Location: 60311 Frankfurt am Main, DE
• Languages: German