Data Engineer

Project overview

Data Engineer

Responsibilities

• Turn agreed DQ rules into automated checks using dbt, SQL and Python
• Build and maintain Airflow DAGs to orchestrate DQ processes
• Embed controls throughout the pipeline life cycle, not just at downstream validation
• Build reusable patterns that apply DQ controls consistently across datasets and data products
• Store DQ results, exceptions and metrics in a structured form for reporting, dashboards and alerting
• Integrate DQ rules and results with the metadata/data catalogue solution so they can be traced
• Implement logging, exception handling, monitoring and alerting

Requirements

If you have a strong data engineering background and have built automated, reusable data quality controls, apply now with your CV.

• Active SC clearance
• Strong SQL for interrogation, validation and reconciliation
• Practical AWS data experience, including S3 and Athena
• Experience implementing DQ controls on cloud data platforms, covering completeness, validity, accuracy, consistency, uniqueness and timeliness
• Experience integrating DQ with metadata management or data catalogues
• Python for automation and integration
• Experience working with Data Engineers, Architects, Governance teams and Data Owners

Project details

5 months (extension likely)

Status: Outside IR35

Remote, with travel to the office as needed

Clearance: Active SC clearance required

Synergetic is recruiting a hands-on Data Quality Engineer for a contract on a modern AWS data platform. Data Governance and business Data Owners define and agree the Data Quality rules. Your job is to build them into automated, reusable controls across ingestion, transformation and data-product pipelines.

• Start: flexible
• Location: London, UK, GB