B2B Project: Data Engineer (Python, Azure)

Project overview

Hays Contracting Services operates under a B2B service delivery model. We are currently looking for a B2B contractor to provide services in the area of Data Engineering and Azure cloud solutions as part of a greenfield data platform initiative focused on regulatory intelligence and data processing.

The project involves building a new data warehouse in the client's Azure environment for a regulatory monitoring platform. As a Mid Data Engineer, you will contribute to the implementation of Python ETL/ELT pipelines, data integrations, and data quality mechanisms, cooperating with a Senior Data Engineer on the target data model and quality standards.

Technology environment:

Scope of services:

Required competencies:

• Python, SQL, ETL/ELT pipelines
• Azure: Data Lake Storage, Data Factory, Functions, Azure SQL
• Databricks or Microsoft Fabric
• REST APIs, JSON/XML, Pydantic, Pandera, Great Expectations
• Git, Azure DevOps, GitHub Actions, Medallion Architecture, Star Schema
• Developing and maintaining production-ready ETL/ELT pipelines in Python
• Building Python-based data pipelines feeding the data warehouse with data from 3-6 regulatory and parliamentary sources across the US, UK, Canada, France, Australia and New Zealand
• Integrating external data sources through REST APIs
• Supporting integrations with non-API data sources, including PDF-based content and web scraping solutions
• Ensuring data quality, validation, monitoring, logging, error handling and reprocessing mechanisms
• Handling schema drift and changes in source data structures
• Working within the client's Azure environment and repositories
• Cooperating with a Senior Data Engineer on implementing the target data model and data quality standards
• Minimum 3 years of commercial experience as a Data Engineer
• Hands-on experience building production ETL/ELT pipelines in Python
• Experience with Azure Data Lake Storage, Azure Data Factory, Azure Functions and Azure SQL
• Practical knowledge of Databricks or Microsoft Fabric
• Strong SQL skills
• Experience working with REST APIs, including authentication, pagination, rate limiting, retry mechanisms and JSON/XML processing
• Experience with data validation frameworks such as Pydantic, Pandera or Great Expectations
• Knowledge of Git and CI/CD practices using Azure DevOps or GitHub Actions
• Understanding of Medallion Architecture and Star Schema concepts
• English at B2+ level, used daily in communication with the international project team
• Ability to work independently, proactively clarify requirements and take ownership of assigned tasks

Nice to have

Cooperation process:

The cooperation is carried out between independent businesses (B2B model) and covers the performance of services within the agreed project scope. The Contractor independently organizes and manages the manner in which the services are performed, taking into account the project requirements, security obligations, and the necessary communication with the client.

Any information regarding location, availability, or project meetings is provided solely for organizational purposes and shall not be construed as creating an employment relationship, employer-like supervision, or performance of work under the direction or control of the client.

#1202141 - Katarzyna Kendzior

• Experience with Terraform or Bicep
• Experience with web scraping solutions
• Experience processing PDF-based data sources
• Experience using AI/LLM capabilities within data pipelines
• Experience designing data models for Power BI reporting environments
• Profile and experience review by the Hays team
• A conversation with a Hays consultant about the project and scope of services
• A technical meeting with project representatives (up to 2 stages)
• Feedback and cooperation proposal
• Welcome to the project!

Project details

Cooperation type: B2B contract

Start date: ASAP

2-3 months (part-time)

145-155 PLN/h (net + VAT)

Experience: min. 3 years

• Start: flexible
• Location: Polska, PL
• Rate: 145-155 EUR/h