Required skills
Job description
Project overview
We are looking for an experienced Data Engineer to support a data platform modernization project involving the migration of existing on-premise solutions to Google Cloud Platform.
The role includes designing and optimizing data pipelines, defining technical solutions, and collaborating with engineers, architects, and business stakeholders to ensure a smooth migration.
- Design and implement data pipeline migration from on-premise environments to GCP.
- Analyze existing workflows and recommend improvements and cloud-native solutions.
- Collaborate with engineers and architects on technical design and implementation.
- Develop and optimize data processing solutions using Spark, PySpark, and Airflow.
- Estimate workloads and report delivery progress and potential risks.
- Work with business stakeholders to clarify requirements and recommend suitable technical solutions.
- Hands-on experience with GCP, particularly BigQuery, IAM, Dataproc, and GCS.
- Knowledge of Terraform and Infrastructure as Code.
- Strong experience with Apache Spark and PySpark (3.5).
- Experience with Apache Airflow (3.x).
- Good SQL skills and experience with data transformation.
- Familiarity with Git, CI/CD, and GitHub Actions.
- Ability to translate business requirements into technical solutions.
- Strong analytical and communication skills.
- Proactive, collaborative approach and fluent English.
Nice to have
- Experience with Scala or Java.
- Familiarity with BI and data visualization tools.
- Previous experience in banking or financial services.
Project details
- Start: ASAP
- Location: Warszawa, PL
Indexed by 9am's meta-search. You are a job provider? Claim or manage this listing
Found on nofluffjobs.com