Data Engineer (Azure)

Project overview

Together with our Partner, a leader in the insurance and reinsurance industry, we are looking for an experienced specialist to provide services as a Data Engineer. The project focuses on the Data Intake area: acquiring data from various sources and automating and ensuring the reliability of pipelines in the Azure and Databricks environment.

WE OFFER:

Thank you for all applications. We would like to inform you that we will contact selected individuals.

13673/DK

• A new way to apply with an AI Recruiter (AIR) – during the process, you can talk to a virtual recruiter 24/7, without waiting for a phone call. The final decision is always made by the Project Manager.
• Cooperation in a project model, with settlement based on stages and acceptance protocols for individual deliverables.
• Support from Connectis in administrative and settlement matters related to cooperation.
• Participation in industry events and technology meetups.
• 5,000 PLN for recommending friends to our projects.
• Flexible B2B cooperation model.
• Service delivery 100% remotely.
• Fast and remote process.
• Setting technical direction and developing engineering best practices in cooperation with other specialists in the project.
• Transforming and preparing data for further use by subsequent layers of the system.
• Scheduling and monitoring pipelines, ensuring proper error handling and logging.
• Building and maintaining pipelines for acquiring data from various sources.
• Ensuring data quality at every stage of the intake process.
• Automating data intake processes in a cloud environment.
• Documented experience in data ingestion and designing reliable pipelines.
• Knowledge of the Azure ecosystem in the area of data processing and storage.
• Practical knowledge of Python in the context of building data pipelines.
• Experience with the Databricks platform in a production environment.
• Fluent knowledge of Polish and English.

Nice to have

• Knowledge of PySpark and experience in large-scale data processing.
• Ability to organize work methods and implement technical standards in the project.
• Senior-level experience allowing for independent technical direction setting.

Project details

• Start: ASAP
• Remote: remote
• Location: Remote, PL
• Rate: 152-155 EUR/h
• Languages: Polish, English