Required skills
- Data Engineering
- Big Data
- Lakehouse
- AWS
- Spark
- PySpark
- Python
- SQL
- dbt
- Apache Iceberg
- AWS Glue
- Amazon EMR
- Amazon S3
- Terraform
- Apache Airflow
Languages: French
Job description
Seeking a Senior/Expert Data Engineer to independently design, develop, industrialize, and operate data products and pipelines within an AWS Lakehouse environment. The role involves ensuring reliable, performant, and maintainable data processing, with a focus on marketing data.
Project overview
Within the Marketing Data team of the Data Fabric unit, you will act as a senior or expert Data Engineer consultant. You will independently take charge of the design, development, industrialization, and operation of data products and pipelines in a Lakehouse environment on AWS. Your objective is to ensure reliable, performant, and maintainable processing, while contributing to the mastery of their daily operation.
You will develop transformations with Spark and PySpark, Python, SQL, and dbt, operate Apache Iceberg data, and orchestrate processing with Airflow. You will contribute to deployment on AWS and infrastructure as code with Terraform. You will implement tests, quality controls, monitoring, and alerts, and participate in incident analysis as well as root cause investigation. In liaison with product managers and business stakeholders, you will translate marketing needs into robust solutions. You will also contribute to documentation, data lineage, data contracts, cataloging, and the team's engineering practices.
Profile
At least 8 years of experience in data engineering and mastery of Big Data and Lakehouse architectures.
Expertise in Spark, dbt, Iceberg, and AWS services, particularly Glue, EMR, and S3.
Proficiency in Python and SQL; knowledge of data modeling tools.
Knowledge of Trino or Starburst and Datadog.
Autonomy, rigor, analytical skills, and understanding of business challenges.
A marketing experience is desired.
Work Environment
The company operates in an environment where digital services and data valorization play an important role. Its teams rely on a data platform to design reliable products and processes, particularly for marketing activities. The mission of this organization and its specific business sector are not detailed in the available information.
In this context, reliability, data quality, and continuous improvement are important operational challenges for the teams.
Project details
- Start: ASAP
- Duration: 10 mois renouvelable
- Remote: hybrid
- Location: Nantes, Pays de la Loire, FR
- Languages: French
Indexed by 9am's meta-search. You are a job provider? Claim or manage this listing
Found on free-work.com