Data Engineer (Security Clearence)

Responsibilities

About you:

• Develop and optimise SQL queries to retrieve, manipulate and analyse complex datasets.
• Build data solutions using Python, including working within notebook-based environments.
• Integrate data from APIs and investigate issues within API responses and associated datasets.
• Work confidently with a variety of data formats, including JSON, CSV and Excel.
• Build, manage and improve automated data pipelines supporting both ETL and ELT approaches.
• Carry out data validation and quality checks to identify inconsistencies, errors and processing issues.
• Diagnose and resolve problems across data pipelines, integrations and datasets.
• Design and maintain database structures that support scalable and reliable data solutions.
• Work within established government security, delivery and data management environments.
• Strong hands-on SQL capability, particularly for querying, transforming and analysing data.
• Commercial experience using Python for data engineering or data processing.
• Experience developing and working within notebook-based environments.
• Good understanding of REST APIs and the ability to interpret and troubleshoot responses.
• Experience handling structured and semi-structured data, including JSON, CSV and Excel.
• Proven experience creating and supporting automated ETL/ELT pipelines.
• Strong understanding of data validation, data quality and troubleshooting.
• Experience designing databases and underlying data structures.
• Active SC clearance is mandatory.
• Must have worked within a UK Government department during the last 12 months.

Project details

• Start: flexible
• Location: London, South East England, GB, GB