Self-Service Data Engineer

Project overview

Senior Full Stack Data Engineer – Azure & Databricks

The Role

Data Engineer role within the Energy sector, working across Data Engineering, Platform Engineering and Quantitative Analytics. You’ll build and productionise scalable data solutions using Azure, Databricks, Python, SQL and Spark, while helping modernise the data platform and turn analytical prototypes into reliable, production-ready data products.

Start date: ASAP

Nice to have

Machine Learning Engineering/MLOps, Terraform or Bicep, monitoring and observability, DataOps, orchestration frameworks, and semantic modelling or dashboard optimisation.

Project details

Rotterdam (hybrid)

Contract Length: 5 months initially (plan to go beyond 12 months)

Contract Type: Freelance

Key Responsibilities

What We’re Looking For

• Design, build and maintain Azure-based data pipelines and data products across batch and streaming workloads.
• Convert Python notebooks and analytical prototypes into robust, tested production solutions.
• Develop, deploy and monitor Azure Databricks workloads using Spark and Delta Lake.
• Collaborate with analysts and quantitative specialists to translate business needs into scalable technical solutions.
• Support data platform migration and modernisation initiatives.
• Improve engineering practices through testing, CI/CD, automation, monitoring and documentation.
• Optimise data transformations and reporting workloads while promoting best practices across the team.
• Strong data engineering experience in Azure environments.
• Hands-on expertise with Azure Databricks, Spark, Delta Lake, Workflows and Unity Catalog.
• Strong Python and SQL skills, with experience building batch and streaming pipelines.
• Experience taking analytical code into production and applying software engineering best practices, including testing, code reviews and CI/CD.
• Familiarity with Azure services such as ADLS Gen2, Azure DevOps, Key Vault and Data Factory.
• Strong stakeholder management skills and the ability to bridge business requirements and technical delivery.
• Experience with platform migration or modernisation.
• Start: flexible
• Location: GB