Skip to content
Job boardOpen

Databricks Ingestion Lead

Confidential · Copenhagen, Denmark · Hybrid · SENIOR

  • Copenhagen, Denmark · Hybrid
  • 4-6 months
  • English

Indexed (today) · refreshed constantly

Start
flexible
Duration
4-6 months
Work mode
Hybrid
Contract
Freelance / B2B
Seniority
SENIOR
Reference
182340

Required skills

Nice to have:Azure Data FactoryData Quality MonitoringPerformance Optimization

Languages: English

Job description

Project overview

This role is responsible for leading the ingestion workstream that reliably transfers all source data, starting with Oracle, into the bronze layer while providing a functional ingestion framework.

Main Responsibilities

Key Requirements

  • Own the ingestion inventory: sources, tables, volumes, load patterns and priorities.
  • Choose the ingestion pattern per source (Lakeflow Connect, Auto Loader, Azure Data Factory) and document the rationale.
  • Design and build a metadata-driven ingestion framework, allowing new sources to be added through configuration rather than new code.
  • Implement Change Data Capture (CDC) and incremental loads, accommodating schema changes, deletions, and late-arriving data.
  • Design for idempotency and replay through checkpoints, watermarks, and controlled reprocessing.
  • Establish monitoring, alerting, load SLAs, and data-quality gates at the bronze layer.
  • Plan and execute cut-over operations for each source, including parallel runs, reconciliation, and sign-offs.
  • Lead and mentor 2-3 data engineers, and ensure proper handover of the framework and runbooks.
  • Extensive experience with Databricks and data ingestion frameworks.
  • Proficiency in Oracle and related data management.
  • Strong understanding of CDC and data load strategies.
  • Experience in metadata-driven architecture design.
  • Ability to lead and mentor engineering teams.

Nice to have

  • Familiarity with Azure Data Factory.
  • Knowledge of data quality monitoring tools.
  • Experience with performance optimization of data ingestion processes.

Project details

ASAP

3-4 months

Copenhagen

  • Start: flexible
  • Duration: 4-6 months
  • Remote: hybrid
  • Location: Copenhagen, Denmark, DK
  • Languages: English

Indexed by 9am's meta-search. You are a job provider? Claim or manage this listing

Found on portal.emagine.org

  1. Job boardNew

    Databricks Architect

    Confidential· Copenhagen, Denmark· Hybrid· Senior

    • Databricks
    • Unity Catalog
    • Terraform
    • +3

  2. Job boardNew

    QA Automation Engineer

    ITFS Sp. z o.o.· Wrocław, Poland· Hybrid· SENIOR· zł 140–160 / h

    • Robot Framework
    • Python
    • Azure Databricks
    • +8

Browse all jobs
Sign up free to apply