Data Engineer

Project overview

We are looking for a data engineer for one of our major clients in Paris.

Start date: Early October

Context:

The Data Chapter, reporting to the Data Directorate, is present at the client's 6 sites.

Its main missions are:

This mission aims to strengthen the Chapter with Data Engineering activities at the Paris site.

• Supporting the company's various squads in the design and implementation of their data use cases, while ensuring the uniformity and consistency of the Data model's application in the creation of use cases.
• Leading data governance, quality, and protection within the company through its Data Office.

Responsibilities

Design and develop pipelines via internal ETL based on Spark / Automate data ingestion from various sources / Orchestrate batch and real-time data flows

Expectations:

Reinforcement of the Data Chapter with an experienced Data Engineer profile who will be involved within a squad of the core business domain, in a Cloudera environment - using an internal company framework – in conjunction with the squad's PO/BA and the Dev team, within an agile framework.

⚠️ Beyond the technical aspects, the profile is expected to be keen and sensitive to business issues: Understand business needs to adapt and deploy the technical solution.

Technical skills:

Language:

Big Data Framework: Hadoop Cloudera (highly appreciated)

Databases (required)

SQL (example: Teradata, Hive, Phoenix)

NoSQL (example: Hbase, MongoDB)

DEVOPS: GIT, Jenkins, Artifactory, XL Release,

Monitoring: ELK Stack (Elasticsearch, Logstash, Kibana), Grafana

Soft skills:

Data governance and protection:

Regulatory compliance (GDPR)

Data lineage and cataloging

• Participate in the definition of the Data architecture (in conjunction with architects and the Squad)
• Refine user stories with the PO/BA, design and then develop solutions for collecting and storing data (from multiple sources) to make it available for business uses:
• Implement unit and automated tests
• Deploy the product in the different environments
• Ensure the proper functioning of the product in production and provide follow-up / support / incident management, including setting up and optimizing monitoring
• Support the evolution of the team's practices in a logic of continuous improvement of code quality
• Spark
• Spark Streaming and batch (required),
• SQL (required)
• xDI

Project details

• Start: ASAP
• Duration: 12 mois (renouvelables)
• Location: 75 - Paris, FR
• Languages: French