Data Ingestion Engineer

Project overview

Our client, a leading AI and robotics business in the automotive space, is looking for a Senior Data Ingestion Engineer who has built data platforms that teach machines to see and move.

We're looking for a Data Ingestion Engineer to build and run the pipelines that bring large, complex datasets into our data platform. You'll work with gigabyte- to terabyte-scale data from many sources and turn it into clean, reliable, well-modelled datasets for analytics, data science and machine learning teams. The role suits an engineer who understands data domains as well as code. You'll know where data comes from, what it means and how it should be shaped for the people who use it.

Key Responsibilities

Essential Skills and Experience

Desirable Experience

This is an Inside IR35 Contract Role working via umbrella company so you must be resident in the UK to be considered, and be prepared to be onsite 2-3 days per week in London Kings Cross.

• Design, build and maintain batch and streaming ingestion pipelines using Python, PySpark and SQL
• Ingest structured, semi-structured and sensor or time-series data from files, APIs, databases and device feeds
• Process and transform datasets from gigabytes to multiple terabytes efficiently and cost-effectively
• Understand the business and technical meaning of each data domain, and model data so it is usable downstream
• Build data quality checks, validation, schema management and monitoring into every pipeline
• Tune Spark jobs for performance, partitioning and storage layout at scale
• Work with data scientists, ML engineers and analysts to understand their needs and deliver the right datasets
• Document pipelines, data lineage and data dictionaries
• Support production pipelines, troubleshoot failures and improve reliability
• Proven commercial experience as a Data Engineer or Data Ingestion Engineer
• Strong Python programming for data processing and pipeline development
• Hands-on PySpark experience with large distributed datasets
• Advanced SQL, including complex queries, performance tuning and data modelling
• Track record of working with large-scale data, from gigabytes up to multiple terabytes
• Good understanding of data domains: how data is produced, what it represents and how it should be structured
• Experience with data quality, validation and schema evolution
• Familiarity with a cloud data platform such as AWS, Azure, GCP or Databricks
• Version control (Git) and CI/CD for data pipelines
• Automotive sector experience, such as vehicle, connected-car, telematics or autonomous driving data
• Working knowledge of IMU (inertial measurement unit) data or other high-frequency sensor and time-series data
• Exposure to computer vision or robotics datasets, such as camera, lidar or sensor-fusion data
• Financial services experience, particularly with large, regulated or high-volume datasets
• Streaming technologies such as Kafka or Spark Structured Streaming
• Lakehouse formats and tooling such as Delta Lake, Apache Iceberg or Parquet
• Orchestration tools such as Airflow or Dagster

Project details

• Start: 2026-11-11 00:00:00
• Duration: 6 Months
• Remote: hybrid
• Location: Greater London, UK, GB
• Languages: English