Tech Lead Data Engineering / Streaming (Apache Flink & Kafka) - >10 years

Project overview

Context: As part of the development and evolution of the Group Data Platform, we are looking for a Tech Lead Data Engineering specialized in real-time architectures, with at least 10 years of experience.

The consultant will be part of the Group Data Delivery team to design, develop, and operate real-time data products, from stream ingestion and processing to their exposure via APIs for internal and external consumers.

They will need to bring advanced technical expertise on Apache Flink, Apache Kafka, event-driven architectures, microservices, and AWS technologies, while ensuring technical leadership, development quality, and reliability of the solutions put into production.

Main missions

• Design and develop end-to-end real-time data products using Apache Flink, Apache Kafka, and event-driven architectures.
• Build and maintain resilient data pipelines for real-time and batch data ingestion, processing, transformation, and distribution.
• Design and develop REST APIs, OpenAPI interfaces, microservices, and streaming interfaces.
• Deploy and operate solutions on AWS, including MSK, Kinesis, Managed Service for Apache Flink, Lambda, API Gateway, ECS, EKS, S3, and IAM.
• Implement data quality controls, data models, validation mechanisms, and schema governance.
• Define and industrialize CI/CD pipelines with GitLab CI, GitHub Actions, or equivalent tools.
• Ensure solution observability using CloudWatch, OpenTelemetry, Prometheus, Grafana, structured logs, metrics, and tracing.
• Ensure the operational maintenance of services: supervision, incident management, root cause analysis, and performance optimization.
• Ensure compliance with SLAs and the availability, security, and reliability of services in production.
• Apply security best practices: OAuth2/OIDC, data encryption, secret management, network control, and the principle of least privilege.
• Write technical documentation: architecture documents, API specifications, operational procedures, and deployment documentation.
• Drive technical choices, lead architecture discussions, and perform code and design reviews.
• Define and promote engineering standards, mentor less experienced developers, and ensure knowledge transfer.
• Ensure the delivery of robust, scalable, secure, and production-ready data products.

Project details

• Start: flexible
• Remote: hybrid
• Location: 92400 Courbevoie, France, FR
• Languages: French