Required skills
- AWS Glue
- Amazon Redshift
- Python
- SQL
- ETL/ELT
- Data Modelling
- Data Quality
- Data Matching
- Record Linkage
- Automated Testing
- CI/CD
Languages: English
Job description
Project overview
Data Engineer - AWS - SC/NPPV3 Cleared
We are seeking an experienced Data Engineer to support the delivery of an AWS-based data solution within a UK Government programme. The role involves building scalable data pipelines, transforming complex datasets, and implementing data-matching and record-linkage solutions.
You will work closely with the Technical Lead and Lead Data Scientist to deliver reliable, production-ready data solutions.
Key Responsibilities
Essential Skills & Experience
- Design, develop and maintain scalable ETL/ELT pipelines using AWS Glue, Redshift, Python and SQL.
- Ingest, transform and validate data from multiple sources and formats.
- Implement data matching, deduplication and record-linkage processes.
- Develop automated testing, monitoring and data quality controls.
- Optimise data pipelines for performance, scalability and cost.
- Apply engineering best practices, including Git, CI/CD and secure development.
- Collaborate with technical teams and stakeholders within an Agile environment.
- Active SC clearance (NPPV3 clearance desirable)
- Strong hands-on experience with AWS Glue, Amazon Redshift, Python and SQL.
- Proven experience building production-grade data pipelines.
- Strong understanding of ETL/ELT, data modelling and data quality.
- Experience with data matching, record linkage or complex data transformations.
- Experience with automated testing, Git and CI/CD.
- Experience working within Agile/Scrum delivery teams.
Project details
- Start: ASAP
- Duration: 3 months
- Location: London, UK, GB
- Rate: 56-63 EUR/h
- Languages: English
Indexed by 9am's meta-search. You are a job provider? Claim or manage this listing
Found on JobServe