Zum Inhalt springen

Diese Ausschreibung ist auf Englisch.

JobbörseOffen

Databricks Architect

Vertraulich · London, Großbritannien · Hybrid · Senior

  • London, Großbritannien · Hybrid
  • English

Indexiert (vor 2 Tagen) · laufend aktualisiert

Budget
78 £ / h
Start
flexible
Arbeitsmodell
Hybrid
Vertrag
Freelance / B2B
Erfahrungslevel
Senior
Referenz
6092269

Geforderte Skills

Nice to have:Microsoft AzureCI/CDGit

Sprachen: English

Jobbeschreibung

Project overview

Job Title: Databricks Architect-Financial Services Client

Project details

London / Manchester (3 Days/Week Onsite)

4 Months+

£620 GBP/day – Inside IR35

Role Overview

We are looking for an experienced Databricks Architect to join a large-scale enterprise data platform programme for a leading Financial Services client.

The ideal candidate will have strong expertise in Databricks, Lakehouse architecture, cloud data platforms and enterprise data engineering, along with experience designing and implementing scalable, secure and high-performance data solutions.

The role will involve providing architectural direction for enterprise Databricks Lakehouse platforms, defining technical standards, designing platform configurations and ensuring effective integration with cloud services such as AWS and Azure.

The successful candidate will work closely with Data Engineering, Cloud, DevOps, Security and Governance teams to deliver reliable, secure and scalable data platform solutions across multiple environments.

Experience working within banking or financial services environments will be highly beneficial.

​

Key Responsibilities

Required Technical Skills

Good to Have

Ideal Candidate Profile

If this position is of interest to you, apply now!

  • Databricks Architecture & Platform Design
  • Define and design enterprise Databricks Lakehouse architecture across Development, Test, UAT and Production environments.
  • Provide architectural direction for Databricks workspace provisioning, compute resources and platform configurations.
  • Define platform architecture standards, design principles and implementation best practices.
  • Design scalable and secure Databricks solutions supporting enterprise data engineering workloads.
  • Guide the selection and configuration of compute resources, cluster policies and job clusters.
  • Ensure platform architecture supports performance, scalability, availability and cost efficiency.
  • Lakehouse Architecture & Data Engineering
  • Design and oversee Lakehouse architecture using Delta Lake.
  • Define data architecture and modelling standards using Bronze, Silver and Gold layers.
  • Provide guidance on batch and incremental data processing pipelines.
  • Oversee the use of Databricks Notebooks, PySpark and SQL transformations.
  • Design and guide the implementation of Databricks Jobs, Workflows and Declarative Pipelines.
  • Define architecture patterns for Databricks Auto Loader and Structured Streaming.
  • Ensure efficient distributed data processing and workload optimisation.
  • Cloud Integration & Infrastructure
  • Design integration between Databricks and cloud platforms such as AWS and Azure.
  • Define architecture for cloud storage, networking, identity and security integrations.
  • Provide guidance on AWS services including S3, IAM, KMS, Glue and Lambda.
  • Design infrastructure provisioning and platform configuration using Terraform and Infrastructure as Code (IaC).
  • Support migration of workloads from AWS-native services to Databricks.
  • Define standards for multi-environment and multi-region platform deployments.
  • Security, Governance & Platform Management
  • Define security architecture and governance standards for Databricks platforms.
  • Design and oversee Unity Catalog implementation, including catalogs, schemas, permissions and access controls.
  • Establish Role-Based Access Control (RBAC), service principals and secrets management.
  • Ensure least-privilege access and secure platform configurations.
  • Define platform monitoring, logging, alerting and operational standards.
  • Ensure compliance with enterprise security and governance requirements.
  • Technical Leadership & Delivery
  • Provide technical leadership and architectural guidance to engineering teams.
  • Work closely with Data Engineering, Cloud, DevOps, Security and Application teams.
  • Review technical designs and ensure alignment with enterprise architecture standards.
  • Guide CI/CD implementation for Databricks configurations, notebooks, jobs and deployments.
  • Drive performance optimisation and platform cost efficiency.
  • Maintain architectural documentation, technical standards and implementation guidelines.
  • Strong experience in Databricks architecture and enterprise Lakehouse platforms.
  • Databricks Notebooks, PySpark and SQL.
  • Delta Lake and Lakehouse architecture.
  • Unity Catalog and data governance.
  • Databricks Jobs, Job Flows and Workflows.
  • Declarative Pipelines and pipeline definitions (DPX files).
  • Databricks Auto Loader and Structured Streaming.
  • AWS services – S3, IAM, KMS, Glue and Lambda.
  • Terraform and Infrastructure as Code (IaC).
  • CI/CD, Git and deployment automation.
  • Distributed data processing and performance optimisation.
  • Multi-environment and multi-region platform architecture.
  • Financial services or banking domain experience.
  • Experience migrating AWS-native workloads to Databricks.
  • Experience with enterprise cloud architecture.
  • Knowledge of Azure cloud services.
  • Experience with platform security hardening and enterprise governance.
  • Experience designing large-scale data platforms.
  • Experience with automated monitoring and operational frameworks.
  • Experienced Databricks Architect with strong enterprise data platform experience.
  • Strong background in designing and implementing Databricks Lakehouse solutions.
  • Hands-on understanding of PySpark, Delta Lake, SQL and Databricks workflows.
  • Experience defining cloud infrastructure, security and governance architecture.
  • Strong knowledge of AWS and Terraform/IaC.
  • Experience providing technical leadership and architectural direction.
  • Ability to work with cross-functional engineering and cloud teams.
  • Financial services or banking experience would be advantageous.
  • Must be comfortable working 3 days per week onsite in London or Manchester.
  • Start: flexible
  • Remote: hybrid
  • Location: London, GB
  • Rate: 78-78 EUR/h
  • Languages: English

Indexiert von der 9am-Meta-Suche. Du bist Job-Anbieter? Listing beanspruchen oder verwalten

Gefunden auf Vallum Associates

  1. JobbörseNeu

    BI Specialist

    Vertraulich· Mannheim, Germany· Hybrid· Senior

    • Power BI
    • SQL
    • Databricks
    • +5

Alle Jobs durchsuchen
Kostenlos anmelden und bewerben