Data Engineer specializing in Data Streaming (Apache Flink / Kafka) - 7+ years

Project overview

Context:

As part of the development and evolution of the Group Data Platform (GDP), the client is looking for a Senior Data Engineer with at least 7 years of experience in Data Engineering, with proven experience in designing and deploying real-time data products.

Integrated into the Group Data Delivery team, the consultant will be involved in the design, development, and operation of data streaming pipelines as well as services for their exposure to internal and external consumers.

They will be responsible for the complete lifecycle of data products, from ingestion and stream processing to availability via APIs, ensuring their quality, security, observability, and reliability in production.

Main missions

• Design and develop end-to-end real-time data products with Apache Flink, Apache Kafka, and event-driven architectures.
• Build and operate resilient pipelines for real-time and batch data ingestion, processing, transformation, and distribution.
• Develop REST APIs, OpenAPI specifications, microservices, and streaming interfaces.
• Operate AWS services: MSK, Kinesis, Managed Service for Apache Flink, Lambda, API Gateway, ECS, EKS, S3, and IAM.
• Implement data quality controls, data models, schema governance, and automated validation processes.
• Design and maintain CI/CD pipelines with GitLab CI, GitHub Actions, or equivalent tools.
• Ensure solution observability using CloudWatch, OpenTelemetry, Prometheus, Grafana, structured logging, metrics, and tracing.
• Ensure operational monitoring of services in accordance with SLAs: supervision, incident management, performance optimization, and root cause analysis.
• Apply security best practices: OAuth2/OIDC authentication and authorization, encryption, secrets management, and the principle of least privilege.
• Write technical documentation, architecture documents, API specifications, and operational procedures.
• Translate business needs into robust and scalable technical designs.
• Participate in architectural choices, perform code reviews, and contribute to defining engineering standards.
• Support less experienced developers through mentoring and knowledge transfer.
• Ensure the delivery of secure, performant, scalable, and production-ready data products.

Project details

• Start: flexible
• Remote: hybrid
• Location: 92400 Courbevoie, France, FR
• Languages: French