Senior Data Engineer / Data Engineering Tech Lead

Project overview

Missions

The Senior Data Engineer / Data Engineering Tech Lead will be responsible for the technical foundations that make data usable, reliable, and scalable 24/7. This senior role is highly operational with light technical leadership responsibilities, not a general .NET development position. The focus is on ingestion, transformation, data quality, traceability, observability, and secure delivery of data products and compute-ready datasets. The role guides technical choices and standards while continuing to design and build critical components directly.

Working within the 24/7 functional team, the Senior Data Engineer / Data Engineering Tech Lead will collaborate with the Product Owner, Business Analysts, software engineers, architects, data platform teams, and country stakeholders. The person must be comfortable delivering tactical solutions without losing sight of the target architecture and reusable capabilities.

Main Responsibilities

A. Data Architecture and Source Integration

B. Time-Series Engineering and Data Modeling

C. Data Transformation and Calculation Activation

D. Data Quality, Traceability, and Observability

E. Engineering, Security, and Operations

Operational Technical Leadership Expected

Priority Data Domains

Tools & Environment

Working Conditions

Skills

• Contribute to end-to-end data architecture and document current and target flows, interfaces, dependencies, and responsibilities.
• Design and build ingestion pipelines from internal datasets, databases, REST APIs, partner feeds, and controlled file exchanges.
• Integrate heterogeneous country data while isolating source-specific logic through reusable connectors and clear interface contracts.
• Implement batch processing and, when justified by business service levels, near real-time or streaming patterns.
• Define robust management for schema evolution, late data, replays, idempotence, backfills, and failed records.
• Collaborate with platform teams to decide when 24/7 should consume existing data products or shared time-series capabilities rather than duplicating local copies.
• Design scalable structures for high-volume interval data, including customer consumption and asset production.
• Define and maintain canonical models for customer, legal entity, site, meter point, contract, asset, allocation, certificate, time series, matching result, and reporting period.
• Apply modeling patterns suitable for application operational data, analytical products, and compute-ready datasets.
• Manage time zones, daylight saving transitions, interval granularity, units of measure, missing intervals, duplicates, and versioned corrections.
• Optimize storage, partitioning, indexing, and processing performance according to loads and retention requirements.
• Build tested transformations that standardize source data into compute-ready consumption and production curves.
• Enable the Hourly Matching Score and associated customer indicators through reproducible and versioned datasets and calculation services.
• Collaborate with business and product specialists to translate calculation rules into explicit technical specifications and automated tests.
• Preserve the distinction between provisional operational measurements and validated or certified reporting results.
• Implement automated checks for completeness, validity, timeliness, uniqueness, consistency, and referential integrity.
• Define reconciliation controls between source, ingested, transformed, and reported data.
• Provide complete technical traceability, execution metadata, and auditable evidence of source, transformation, and calculation versions.
• Create monitoring, alerting, and operational dashboards covering freshness, pipeline failures, rejected records, and service level adherence.
• Contribute to incident analysis, root cause correction, and recurrence prevention.
• Utilize Git-based development, peer review, automated testing, and CI/CD to deliver reliable data pipelines.
• Contribute to Infrastructure as Code and environment configuration with DevOps and platform teams.
• Apply security by design: least privilege access, secret management, encryption, environment separation, and secure processing of customer data.
• Produce maintainable code, runbooks, architecture decisions, data contracts, and operational documentation.
• Participate in Agile ceremonies while maintaining a technical backlog for reliability, scalability, and technical debt.
• Act as a senior data engineering reference within the 24/7 squad while remaining directly involved in design, coding, testing, and production support.
• Define pragmatic engineering patterns for ingestion, transformation, testing, deployment, monitoring, and recovery, ensuring their consistent adoption.
• Lead technical and code reviews; coach software engineers contributing to data flows without becoming a manager.
• Challenge tactical duplications and formulate explicit build-versus-reuse recommendations with B2B platform and architecture stakeholders.
• Transform ambiguous requirements and fragmented sources into sequenced technical decisions, deliverable increments, and a controlled technical backlog.
• Own key non-functional decisions covering scalability, reliability, security, operability, maintainability, and cost.
• Customer reference data, legal entity, site, and delivery point.
• Contractual parameters, target matching commitments, and reporting schedules.
• Hourly or sub-hourly customer consumption data and meter data.
• Identity, characteristics, and production data of renewable or carbon-neutral assets.
• Allocation of assets and energy volumes to contracts.
• Certificates or Guarantees of Origin and associated evidence.
• Matching results, methodological versions, and customer reporting outputs.
• Advanced Python and SQL for production data processing
• Data pipelines integrating APIs, databases, and files
• Git, code review, CI/CD, and environment promotion
• Cloud data engineering on Azure or AWS
• Logging, monitoring, alerting, and incident management tools
• Data access security and secret management
• Databricks, Spark, Delta Lake, Unity Catalog (highly appreciated)
• Time-series specialized platforms like PostgreSQL / TimescaleDB
• Data Lake / Lakehouse architecture
• Real-time or streaming services (Kafka, Event Hubs, or equivalent)
• Terraform or other Infrastructure-as-Code tools
• Data cataloging, traceability, and governance tools
• Power BI or semantic layer integration
• AI-assisted engineering tools in a secure framework
• Contract and details not specified
• Location not specified
• Collaboration with international and multi-country teams
• Agile environment with strong business and technical interaction
• English: Junior (< 3 years)
• AWS: Junior (< 3 years)
• Python V3: Junior (< 3 years)
• SQL: Junior (< 3 years)

Project details

• Start: flexible
• Remote: hybrid
• Location: Paris, France, FR
• Languages: French