Skip to content
Job boardOpen

Spark / Data Lakehouse Developer

Ideamotive · Wrocław, -, Poland · Hybrid · Senior

  • Wrocław, -, Poland · Hybrid
  • Polish

Indexed (2 days ago) · refreshed constantly

Budget
zł 125–145 / h
Start
flexible
Work mode
Hybrid
Contract
Freelance / B2B
Seniority
Senior
Reference
ideamotive-spark-data-lakehouse-developer-wroclaw-data-5592f2fd

Required skills

Nice to have:Delta LakeApache HudiKanbanJiraConfluence

Languages: Polish

Job description

We are looking for an experienced specialist for the Data Platform team to build and maintain data layers in a Data Lakehouse architecture. The role involves remote work with occasional office visits.

Project overview

We are looking for an experienced specialist for the Data Platform team for one of our clients in the financial sector. The project focuses on building and maintaining data layers in a Data Lakehouse architecture. This includes the full processing cycle, from raw datasets to presentation layers (curated, silver, gold). The solutions are based on a modern technology stack, including Apache Iceberg and MongoDB in the ODS (Operational Data Store) layer.

Remote work, with one day per month visit to the office in Wrocław.

Your responsibilities

What we expect

  • Designing and implementing data reconciliation solutions in the MongoDB-based ODS layer.
  • Synchronizing data from source systems.
  • Building and maintaining data layers in a Data Lakehouse architecture (Delta Lake, Apache Iceberg, Hudi).
  • Managing versioned tables and optimizing writes (upserts, merge) in Apache Iceberg.
  • Automating the build, testing, and deployment processes of Spark applications using Jenkins.
  • Monitoring solutions using Prometheus and Grafana tools.
  • Minimum 4 years of practical experience with Apache Spark in production environments.
  • Proficient knowledge of PySpark, Spark SQL, DataFrames, and Structured Streaming.
  • Experience working with MongoDB, including reading and writing data via Spark Connector, document modeling, and indexing.
  • Practical knowledge of Data Lakehouse architecture and tools such as Apache Iceberg.
  • Ability to create CI/CD processes using Jenkins.
  • Proficiency in Python programming.
  • Knowledge of system monitoring issues (Prometheus, Grafana).
  • Experience working with agile methodologies (Scrum or Kanban) and knowledge of Jira and Confluence.

Project details

  • Start: flexible
  • Remote: hybrid
  • Location: Wrocław, -, PL
  • Rate: 125-145 EUR/h
  • Languages: Polish

Indexed by 9am's meta-search. You are a job provider? Claim or manage this listing

Found on justjoin.it

  1. Job boardNew

    AWS Data Engineer

    Confidential· India· On-site· Senior

    • AWS
    • Amazon S3
    • AWS Glue
    • +9

Browse all jobs
Sign up free to apply