Required skills
- Databricks
- Apache Spark
- Delta Lake
- Apache Kafka
- Medallion Architecture
- Azure Data Factory
- Microsoft Fabric
- Metadata-driven Ingestion Frameworks
- Semantic Data Modeling
- Event-driven Data Architecture
Languages: English · Portuguese
Job description
Project overview
Assignment Description
We are seeking a skilled Data Architect / Team Leader to design and implement scalable data pipelines using Databricks (Spark + Delta Lake) across both batch and streaming workloads. This role includes integrations with Kafka and medallion architecture patterns (Bronze/Silver/Gold).
The successful candidate will:
- Develop and orchestrate metadata-driven ingestion frameworks.
- Manage custom business logic transformations through Databricks notebooks and workflow orchestration.
- Build and maintain semantic data models to support Power BI analytics.
- Ensure reliable, governed datasets for business reporting.
- Work with event-driven data architectures.
- Utilize cloud orchestration tools such as Azure Data Factory (ADF) for automated data ingestion and processing.
Requirements
- Strong experience with Databricks, Spark, and Delta Lake.
- Familiarity with Kafka and medallion architecture patterns.
- Proficiency in developing metadata-driven ingestion frameworks.
- Experience in building semantic data models for analytics.
- Knowledge of event-driven data architectures.
- Working knowledge of Azure Data Factory (ADF).
- Experience in migration to Fabric.
- Experience in AI is a strong plus.
Project details
This project is a remote opportunity with very occasional trips to the offices in Lisbon, allowing for flexible work arrangements. Ideal for professionals aiming to work on cutting-edge data architecture initiatives.
- Start: flexible
- Duration: > 12 months
- Remote: hybrid
- Location: Lisbon, Portugal, PT
- Languages: English, Portuguese
Indexed by 9am's meta-search. You are a job provider? Claim or manage this listing
Found on portal.emagine.org