Senior Data Engineer - Azure Databricks & PySpark (80-100%)

Project overview

Job description:

About the customer:

In this role, you will design and build scalable data pipelines, data products, and Back End services using Azure Databricks, Apache Spark, PySpark, Python, and SQL. You will contribute to a modern cloud-based data architecture and help deliver reliable, high-performance data solutions.

Framework;

• Design, develop, and maintain scalable data pipelines, data products, and Back End services supporting analytics, reporting, Real Time processing, and AI use cases.
• Build and optimize large-scale data processing solutions using Databricks, Apache Spark, PySpark, Spark SQL, and Delta Lake.
• Develop and maintain efficient ETL/ELT pipelines and data transformation processes.
• Design and implement data models for analytical and operational workloads.
• Write high-quality, secure, maintainable, and well-tested code using Python, Scala, and SQL.
• Implement and enhance data quality, governance, and monitoring solutions.
• Contribute to CI/CD processes, automated deployments, and modern engineering practices.
• Collaborate closely with cloud, security, architecture, and business teams.
• Location: Zurich, Switzerland
• Start Date: ASAP (flexible for the right candidate)
• Contract Duration: 12 months, with possible extension
• Work Model: Initially 100% onsite in Zurich during onboarding, followed by a hybrid setup (60% onsite/40% remote within Switzerland)

Requirements

• 5+ years of professional experience in Data Engineering, including building enterprise-scale data platforms and ETL/ELT solutions.
• Strong hands-on experience with Apache Spark, PySpark, and/or Spark Streaming (must be clearly demonstrated in your CV).
• Proven experience working with Azure Databricks and large-scale data processing.
• Excellent programming skills in Python and SQL, including SQL performance optimization and troubleshooting.
• Experience with Spark SQL, Structured Streaming, and Delta Lake.
• Good understanding of data modelling, data warehousing, and modern data platform architectures.
• Experience with cloud-based data solutions, preferably Microsoft Azure.
• Familiarity with CI/CD pipelines, automated testing, and deployment practices.
• Strong analytical and problem-solving skills, with the ability to work independently and collaboratively.
• Excellent communication skills and experience working in Agile environments.

Nice to have

• Experience in financial services, particularly Structured Products, Trading, or Capital Markets.
• Experience with Terraform and Infrastructure as Code (IaC).
• Knowledge of streaming technologies such as Kafka, Azure Event Hubs, Event Grid, or Apache Flink.
• Experience with dbt or Data Vault 2.0.
• Experience providing technical guidance and contributing to architectural decisions

Project details

• Start: flexible
• Location: Zürich, Switzerland, CH

Source notes

• A challenging 12-month contract with the possibility of extension.
• The opportunity to work on a modern Azure Databricks Data Platform within a leading financial institution.
• Exposure to innovative Data Analytics and AI initiatives.
• A collaborative environment with experienced engineering, cloud, and architecture professionals.
• A hybrid working model following the initial onboarding period.
• A straightforward recruitment process consisting of two interview rounds.