Zum Inhalt springen

Diese Ausschreibung ist auf Englisch.

JobbörseOffen

Data Architect

Vertraulich · London, Großbritannien · Senior

  • London, Großbritannien
  • English

Indexiert (vor 2 Tagen) · laufend aktualisiert

Start
flexible
Vertrag
Freelance / B2B
Erfahrungslevel
Senior
Referenz
6092246

Geforderte Skills

Nice to have:GitCI/CDS3IAMKMSAWS GlueAWS LambdaTerraform

Sprachen: English

Jobbeschreibung

Project overview

The Role

You will build, secure, automate, and run enterprise Databricks Lakehouse platforms for banking and financial services clients. The role covers workspace provisioning, infrastructure-as-code, CI/CD, security hardening, and platform operations across multiple environments and regions.

Responsibilities

  • Build, configure, and maintain enterprise Databricks Lakehouse platforms across Dev, Test, UAT, and Production environments.
  • Provision and manage Databricks workspaces, compute resources, cluster configurations, and platform-level settings.
  • Automate infrastructure provisioning and configuration using Terraform / Infrastructure as Code (IaC).
  • Develop and maintain CI/CD pipelines for Databricks platform configurations, notebooks, jobs, and application deployments.
  • Configure and administer Unity Catalog, including catalogs, schemas, permissions, and access controls.
  • Implement RBAC, service principals, secrets management, and least-privilege access for secure platform operations.
  • Integrate Databricks with cloud services such as AWS or Azure, including storage, networking, identity, and security services.
  • Configure and maintain cluster policies, job clusters, compute configurations, and environment standards.
  • Monitor platform health, workloads, jobs, and compute resources and troubleshoot production issues.
  • Implement logging, monitoring, alerting, and operational processes to improve platform reliability.
  • Optimize Databricks workloads and compute configurations for performance and cost efficiency.
  • Automate repetitive platform administration, deployment, and operational activities.
  • Support multi-environment and multi-region deployments, ensuring consistent platform configurations.
  • Work with Data Engineering, Cloud, DevOps, Security, Governance, and Application teams on platform integration.
  • Maintain technical documentation, deployment standards, operational procedures, and platform configurations.
  • Support security hardening and governance requirements appropriate for banking and financial-services environments.

Requirements

Essential skills/knowledge/experience:

Databricks & Lakehouse

If this position is of interest to you, apply now!

  • Strong hands-on experience with:
  • Databricks Notebooks
  • PySpark DataFrame API
  • SQL transformations
  • Delta Lake
  • Databricks Jobs, Job Flows, Workflows
  • Declarative Pipelines (dpx files, pipeline definitions)
  • Experience with Unity Catalog
  • Databricks Autoloader and Structured Streaming
  • Terraform or IaC experience for Databricks configuration (Optional)
  • Experience building both batch and incremental pipelines.
  • AWS Stack
  • Strong familiarity with:
  • S3, IAM, KMS
  • Glue / Lambda
  • Experience migrating workloads from AWS native tools to Databricks.
  • General Data Engineering
  • Distributed data processing and performance tuning.
  • Strong knowledge of Lakehouse medallion architecture
  • Data modeling for Lakehouse: bronze → silver → gold patterns.
  • Experience with Git, CI/CD.

Project details

  • Start: flexible
  • Location: London, GB
  • Languages: English

Indexiert von der 9am-Meta-Suche. Du bist Job-Anbieter? Listing beanspruchen oder verwalten

Gefunden auf Vallum Associates

  1. JobbörseNeu

    BI Specialist

    Vertraulich· Mannheim, Germany· Hybrid· Senior

    • Power BI
    • SQL
    • Databricks
    • +5

Alle Jobs durchsuchen
Kostenlos anmelden und bewerben