Data Engineer

Project overview

Role: Data Engineer

IR35 determination: Inside IR35

Responsibilities

---

• Design, implement, and maintain high-throughput data pipelines using Apache Spark and cloud-native tools
• Develop ETL/ELT workflows for managing structured and unstructured data in AWS cloud environments
• Collaborate with AI/ML teams to enable data platforms for Generative AI and advanced analytics use cases
• Ensure compliance with data quality, governance, and monitoring best practices across all integrations
• Enable data lakehouse architectures that support scalability, resilience, and flexible analytics
• Write optimized SQL queries and scalable code in Java or Scala, ensuring performance efficiency in large datasets
• Apply security principles and operational guidelines to deliver secure, highly available solutions
• Partner with cross-functional engineering teams to improve automation, data observability, and platform reliability
• Troubleshoot complex data processing issues within distributed systems and production environments
• Contribute to continuous improvement strategies for AI-ready data pipelines and analytics enablement

Requirements

---

• 5+ years of data engineering experience building high-performing and large-scale data solutions
• Hands-on expertise in Apache Spark development using Java or Scala (Python experience considered)
• Strong proficiency in AWS cloud services (S3, EMR, Glue, IAM, etc.) and cloud-native architecture principles
• Advanced SQL development skills for performance-critical data transformations
• Practical knowledge of ETL/ELT pipelines, data modelling, and lakehouse design in an enterprise environment
• Familiarity with Generative AI or AI/ML workflows and experience enabling AI-driven data ecosystems
• Solid understanding of data governance, quality, security standards, and operational resiliency

Nice to have

• Hands-on experience with Databricks and/or Snowflake, ideally within modern cloud data platforms
• Financial services domain experience, with exposure to data compliance frameworks
• Familiarity with observability tools, automation practices, and modern DevOps principles in data workflows

Project details

Birmingham

Working model: Hybrid 3 days on site each week

up to £475 per day

12 months

---

• Start: ASAP
• Duration: 12 months
• Location: Birmingham, UK, GB
• Rate: 59-59 EUR/h