Zum Inhalt springen

Diese Ausschreibung ist auf Englisch.

JobbörseOffen

Databricks Ingestion Lead

Vertraulich · Copenhagen, Denmark · Hybrid · SENIOR

  • Copenhagen, Denmark · Hybrid
  • 4-6 months
  • English

Indexiert (heute) · laufend aktualisiert

Start
flexible
Laufzeit
4-6 months
Arbeitsmodell
Hybrid
Vertrag
Freelance / B2B
Erfahrungslevel
SENIOR
Referenz
182340

Geforderte Skills

Nice to have:Azure Data FactoryData Quality MonitoringPerformance Optimization

Sprachen: English

Jobbeschreibung

Project overview

This role is responsible for leading the ingestion workstream that reliably transfers all source data, starting with Oracle, into the bronze layer while providing a functional ingestion framework.

Main Responsibilities

Key Requirements

  • Own the ingestion inventory: sources, tables, volumes, load patterns and priorities.
  • Choose the ingestion pattern per source (Lakeflow Connect, Auto Loader, Azure Data Factory) and document the rationale.
  • Design and build a metadata-driven ingestion framework, allowing new sources to be added through configuration rather than new code.
  • Implement Change Data Capture (CDC) and incremental loads, accommodating schema changes, deletions, and late-arriving data.
  • Design for idempotency and replay through checkpoints, watermarks, and controlled reprocessing.
  • Establish monitoring, alerting, load SLAs, and data-quality gates at the bronze layer.
  • Plan and execute cut-over operations for each source, including parallel runs, reconciliation, and sign-offs.
  • Lead and mentor 2-3 data engineers, and ensure proper handover of the framework and runbooks.
  • Extensive experience with Databricks and data ingestion frameworks.
  • Proficiency in Oracle and related data management.
  • Strong understanding of CDC and data load strategies.
  • Experience in metadata-driven architecture design.
  • Ability to lead and mentor engineering teams.

Nice to have

  • Familiarity with Azure Data Factory.
  • Knowledge of data quality monitoring tools.
  • Experience with performance optimization of data ingestion processes.

Project details

ASAP

3-4 months

Copenhagen

  • Start: flexible
  • Duration: 4-6 months
  • Remote: hybrid
  • Location: Copenhagen, Denmark, DK
  • Languages: English

Indexiert von der 9am-Meta-Suche. Du bist Job-Anbieter? Listing beanspruchen oder verwalten

Gefunden auf portal.emagine.org

  1. JobbörseNeu

    Databricks Architect

    Vertraulich· Copenhagen, Denmark· Hybrid· Senior

    • Databricks
    • Unity Catalog
    • Terraform
    • +3

  2. JobbörseNeu

    QA Automation Engineer

    ITFS Sp. z o.o.· Wrocław, Polen· Hybrid· SENIOR· 140–160 zł / h

    • Robot Framework
    • Python
    • Azure Databricks
    • +8

Alle Jobs durchsuchen
Kostenlos anmelden und bewerben