Required skills
- Python
- AWS
- PostgreSQL
- PySpark
- ETL Pipeline Development
- AWS Lambda
- AWS Glue
- AWS RDS
- AWS Step Functions
- AWS Athena
- AWS CDK
- Infrastructure as Code
- GitLab CI
Languages: German
Job description
Seeking an experienced Data Engineer with a focus on cloud technologies and data processing for a project in the transport and logistics sector. The role involves developing and operating a cloud application on AWS, Python, and PostgreSQL.
Project overview
For a project in the transport and logistics sector, we are looking for an experienced Data Engineer (m/f/d) with a focus on cloud technologies and data processing.
Responsibilities
You will develop and operate a cloud application based on AWS, Python, and PostgreSQL. Your responsibilities include the conception and implementation of ETL pipelines, connecting data sources via APIs or cross-account access, developing automated tests (unit and integration tests), and data quality tests. Furthermore, you will take care of monitoring, logging, and alerting. You will build and maintain CI/CD pipelines with GitLab CI and implement security scanners.
Requirements
You bring at least 3 years of experience in Python software development, including testing and code quality. Extensive experience in AWS cloud development (Lambda, Glue, RDS, Step Functions, Athena) is required. You have proven expertise in developing ETL pipelines with Python/PySpark and experience with Infrastructure as Code using AWS CDK. Knowledge of GitLab CI and CI/CD processes is necessary.
Advantageous are experiences in cloud architecture conception, PostgreSQL/AWS RDS, AWS IAM, and data governance, as well as knowledge of rail transport.
Framework:
The project starts on 01.01.2027 and runs for 12 months. The work location is Frankfurt am Main with a 95% remote option. The position is designed for freelancers.
We look forward to your application with CV and hourly rate.
Project details
- Start: 01.01.2027
- Duration: 12 Monate
- Remote: hybrid
- Location: Frankfurt am Main, DE
- Languages: German
Indexed by 9am's meta-search. You are a job provider? Claim or manage this listing
Found on nexus.webconia.de