Required skills
- Microsoft Fabric
- OneLake
- Apache Spark
- PySpark
- Delta Lake
- Direct Lake
- Microsoft Entra ID
- Azure RBAC
- Azure Key Vault
- Python
- CI/CD
- Medallion architecture
Languages: French
Job description
Seeking a senior Data Engineering Architect with expertise in Microsoft Fabric to audit and optimize the existing data platform, focusing on security, development foundations, and performance/cost/sustainability.
Project overview
Context
The bank is organized into five business lines: Corporate Banking, Banking for Regulated Professions, Wealth Management, Corporate & Investment Banking, Correspondent Banking. Its Data BI Analytics team has 9 people reporting to the Head of Data (product manager, analytics engineers, data engineers, data analysts) and develops with agents (Claude Code, Copilot).
The platform has been running in production on Microsoft Fabric for a year. It feeds reporting (Net Banking Income, business cockpits, data quality) and regulatory projects. The team wants to stabilize this foundation before adding the Fabric IQ ontology and AI agents.
Environment
Missions
The expert will audit and optimize the foundation on three fronts. Each will be measured before and after, then transmitted to the team (documentation, best practices, restitution workshop). The focus will be on the most costly processes rather than reviewing everything.
Security and Permissions
Development Foundation
Performance, Cost, and Sustainability
Expected Skills
Profile
8+ years in data engineering or data architecture, with a Fabric deployment conducted in production.
- Fabric: OneLake, Lakehouse, Spark / PySpark, SQL endpoint, Direct Lake, dedicated capacity
- Medallion architecture Bronze / Silver / Gold, _ops chain
- Approximately 180 pipelines, 600 notebooks, a shared Python package
- Delta Lake and Parquet formats
- Sources: SAB core banking, Salesforce, via a real-time API layer and Fabric DataHub
- Security: Entra ID, Fabric roles, RLS / OLS, Azure RBAC
- Delivery: Power BI, interactive HTML applications
- Review of workspaces, roles, Entra ID groups, shares, RLS / OLS, sensitivity labels, tenant settings, identities, shortcuts, private endpoints
- Access mapping, prioritized gap report, target permissions matrix, Fabric / Azure RBAC matrix
- Remediation plan, naming standard, recertification procedure
- Access governance via OneLake Catalog and OneLake Security, anonymization architecture outside of production
- Inventory of formats by layer and Delta / Parquet doctrine, with decision grid and convergence plan
- Diagnosis and optimization of daily loads exceeding 30 minutes
- Audit of the most costly jobs: Spark pools, executors, autoscale, partitions, shuffle, AQE, small files
- Partitioning doctrine, retention policy, OPTIMIZE / VACUUM maintenance plan
- Code review (anti-patterns, tests, documentation), quantified quick wins, pilot job refactoring
- Preparation of tables for Direct Lake
- Microsoft Fabric: Lakehouse, OneLake, Spark / PySpark, Delta Lake, SQL endpoint, Direct Lake, pipelines, environments, capacity management. DP-600 or DP-700 certification appreciated
- Azure: Entra ID, RBAC, Key Vault, storage, private endpoints
- Software Engineering: Packaged Python (wheel), automated tests, Git, CI/CD, code review
- Data: Medallion architecture, modeling, data quality, Power BI semantic models
- AI Practices: Common use of Claude Code or Copilot
- Business: Banking context and accounting data appreciated, knowledge of SAB is a plus
Project details
- Start: flexible
- Remote: hybrid
- Location: Paris, France, FR
- Languages: French
Indexed by 9am's meta-search. You are a job provider? Claim or manage this listing
Found on collective.work