Job description
Project overview
Lead/Principal Data Engineer - Python, SQL, Pyspark, Databricks - Inside IR35
Currently engaged by an industry leading client that are seeking a Lead/Principal Data Engineer with Python, SQL, Pyspark and Databricks experience.
- Strong Python and SQL
- PySpark and Databricks
- Customer identity resolution/entity matching
- Deterministic, probabilistic or model-based matching
- Data modelling and Customer 360/Golden Record concepts
- Large-scale data processing
- Data quality and matching-quality measurement
- Feature engineering
- Productionisation of analytical/ML logic
- Ability to design the solution architecture and lead implementation
Nice to have
Role expectations
Lead/Principal Data Engineer - Python, SQL, Pyspark, Databricks - Inside IR35
- Graph/entity-resolution techniques
- Record linkage and fuzzy matching
- ML-based entity resolution
- MDM/Customer Master experience
- Match/merge and survivorship concepts
- MLOps and feature stores
- Design and implement scalable customer identity-resolution solutions
- Build governed customer hierarchies and Golden Records
- Develop reusable deterministic and probabilistic matching frameworks
- Define matching-quality and data-quality controls
- Integrate identity-resolution outputs into curated data layers
- Drive architecture, technical standards and production readiness
Project details
- Start: ASAP
- Duration: 6 Months
- Location: London, UK, GB
- Rate: 50-59 EUR/h
Indexed by 9am's meta-search. You are a job provider? Claim or manage this listing
Found on JobServe