Required skills
- Data Exploration
- Data Analysis
- Data Modeling
- Data Quality
- Data Stewardship
- Software Development
- Version Control
- CI/CD
- Automated Testing
- Power BI
- ETL
- SQL
- Python
- AWS Cloud Technologies
- AWS Managed Services
- Data Pipelines
- AWS Glue Data Quality
- German Language
Languages: German
Job description
Seeking a Senior Data Engineer for a cloud-native Big Data and Machine Learning environment focused on analyzing vehicle data. The role involves developing and automating data pipelines, ensuring data quality, and supporting data science use cases.
Project overview
For our client, a major corporation in the rail and logistics sector, we are looking for support on the following project:
The expert environment is a cloud-native end-to-end environment for Big Data and Machine Learning. It enables the analysis of vehicle data by linking technical and business data sources. Components include unified data management for Data Engineers, an analysis environment for Data Scientists, and self-service monitoring and dashboarding for BI analysts. Users come from the digital maintenance sector and focus on developing use cases and technically analyzing equipment.
This includes, for example: assessing the frequency of error messages for specific components of a particular series or predicting component failures based on sensor data. Another use case is assessing the business impact of downtime measures for specific weaknesses. For this purpose, error frequencies are combined with the associated costs of error correction at the individual maintenance locations. The analysis results are made available to further consumers, e.g., the respective business units or organizational units.
Services:
Requirements – must have (mandatory, also listed and explained in years and individual project references!)
Requirements – nice to have
I look forward to receiving your profile including availability and desired terms!
- Development and automation of ETL/ELT pipelines and data models using SQL, Python, and Spark or PySpark.
- Use of AWS Managed Services for data ingestion, transformation, orchestration, and provisioning.
- Ensuring data quality and data provision for analyses and dashboards, as well as support for data science use cases.
- Implementation of modern software development and CI/CD practices, as well as collaboration in an agile team.
- Experience with Data Exploration, Data Analysis, Data Modeling, and Data Quality or Data Stewardship (min. 3 years)
- Experience in software development with version control, CI/CD, and automated testing (min. 3 years)
- Project experience with Power BI as a BI and dashboard solution (Number of references: 1)
- Relevant experience in the development and automation of ETL pipelines and in data modeling with SQL and Python, including at least three years with AWS cloud technologies and managed services for data pipelines (min. 5 years)
- Project experience with the AWS service AWS Glue Data Quality and the creation of Glue Data Quality rules (Number of references: 1)
- German: Mandatory
- Experience with serverless cloud data platforms or a comparable modern cloud data architecture (Number of references: 1)
- Experience with established data science methods for forecasting, classification, or clustering (Number of references: 1)
- Practical experience with Docker containers (min. 3 years)
- Experience with transport and logistics data as well as heterogeneous data sources, data formats, and quality requirements (up to 3 years)
- Experience in partially taking on Product Owner tasks, especially requirements gathering and prioritization, formulation of user stories, backlog maintenance, or sprint planning (Number of references: 1)
Project details
Period: 01.01.2027 – 31.12.2027 + Option for extension
94% Remote / 6% onsite in Frankfurt am Main
Project Language: German C1/C2
Utilization: 200 PT
- Start: 2027-01
- Duration: 12 Monate
- Remote: remote
- Location: Frankfurt am Main, Deutschland, DE
- Languages: German
Indexed by 9am's meta-search. You are a job provider? Claim or manage this listing
Found on FreelancerMap DE