DataBricks Engineer

Project overview

Databricks Data Engineer

Title: Databricks Data Engineer

Contract: Initial 6-month contract

Project details

Up to € 300 daily

Madrid - 2 days a week on site

Key Responsibilities

Mandatory Skills & Experience

Ideal Candidate

The ideal candidate will be a hands-on Data Engineer who can take ownership of data pipeline development, from source ingestion through to gold layer delivery, with a strong focus on scalability, performance, data quality and maintainability.

• Design and develop end-to-end data pipelines, from source systems through to Gold layer datasets.
• Implement data ingestion, transformation and optimisation using Databricks.
• Build scalable and efficient data processing pipelines using PySpark.
• Ingest and process data from multiple sources, including APIs, databases and files.
• Implement and maintain Medallion architecture across Bronze, Silver and Gold layers.
• Develop and optimise data transformations using SQL and Python.
• Process large-scale datasets efficiently while maintaining data quality and reliability.
• Ensure pipelines are maintainable, scalable and optimised for performance.
• Strong hands-on experience with Databricks.
• Advanced PySpark development skills.
• Strong Python programming experience.
• Proven experience building end-to-end data pipelines.
• Expertise in data ingestion from multiple sources, including APIs, databases and files.
• Experience implementing Medallion architecture (Bronze, Silver and Gold layers).
• Strong SQL skills and experience in data transformation.
• Experience handling large-scale data processing.
• Work Type: Contract
• Salary/Rate: € 250-300 per day
• Remote Work: No
• IR35 Status: Not Applicable
• Start: flexible
• Remote: onsite
• Location: Madrid, GB
• Rate: 31-38 EUR/h