Senior Data Engineer- HealthCare Industry

Project overview

Important: Candidates must have recent experience in the healthcare industry

Key Responsibilities

• Analyse and profile raw manufacturing and operational data from the external site, identifying source-system structures, data-quality issues, dependencies, and onboarding requirements.
• Design, build, test, and maintain scalable Python-based Databricks ETL notebooks and pipelines for ingestion, transformation, validation, and publishing.
• Create reusable CDL bronze and silver asset-centric datasets that conform to enterprise data architecture, engineering, metadata, security, and quality standards.
• Develop a flexible master data framework that consistently aligns products, materials, equipment, processes, batches, and relevant manufacturing reference data.
• Build governed KPI calculation frameworks for yield, cycle time, process deviations, and other agreed product and manufacturing-process performance measures.
• Develop aggregated performance models that support trend analysis and reporting over time, enabling insights into operational efficiency and product performance.
• Partner with manufacturing, quality, business, analytics, and source-system subject-matter experts to define requirements, validate business rules, and confirm the fitness of delivered datasets and metrics.
• Implement automated data-quality checks, reconciliation, monitoring, logging, and exception handling to ensure reliable and traceable data products.
• Optimise pipeline performance, reusability, and maintainability, and document source mappings, transformation logic, data lineage, KPI definitions, and operational support procedures.
• Work within agile delivery and DevOps practices, including source control, peer review, automated testing, release management, and production support.
• Provide technical leadership, engineering guidance, and mentoring while promoting consistent standards and reusable patterns across the delivery team.

Requirements

• Bachelor's degree in Computer Science, Data Engineering, Information Technology, Engineering, or a related discipline; equivalent relevant experience will also be considered.
• Typically 7 or more years of relevant data engineering experience, including ownership of complex data onboarding and modelling deliveries.
• Advanced hands-on Python and SQL skills, with strong software-engineering practices for modular, testable, supportable data solutions.
• Strong practical experience with Databricks, Apache Spark, notebook-based development, Delta Lake or equivalent lakehouse patterns, and performance optimisation.
• Demonstrated experience designing scalable ETL/ELT pipelines and layered bronze and silver data products using reusable engineering patterns.
• Experience designing master data, dimensional, asset-centric, aggregation, and time-series models for analytics and reporting.
• Strong understanding of data quality, metadata, lineage, reconciliation, observability, access controls, and production support.
• Experience working with manufacturing and operational data, including batch, product, material, equipment, process, quality, and performance data.
• Experience with manufacturing execution systems (MES), SAP, historians, laboratory or quality systems, and industrial analytics environments is highly desirable.
• Knowledge of manufacturing KPIs and the underlying business rules for yield, cycle time, deviations, process performance, and trend-based analysis is highly desirable.
• Experience with source control, CI/CD, automated testing, agile delivery, and regulated or validated data environments is preferred.
• Strong analytical, problem-solving, stakeholder-management, and written and verbal communication skills, with the ability to translate operational requirements into robust technical solutions.
• Proven ability to work across internal teams and external-site stakeholders, influence technical decisions, and deliver effectively with limited supervision.
• Core technical capabilities: Python, SQL, Databricks, Apache Spark, lakehouse and Delta patterns, ETL/ELT pipeline engineering, data modelling, master data, data quality, metadata and lineage, Git-based source control, CI/CD, and cloud data-platform delivery.

Project details

• Start: flexible
• Duration: 6-12 months
• Location: United Kingdom, GB
• Rate: 31-31 EUR/h