Data Engineer - Senior

Project overview

The Data Engineer creates the data infrastructure in a simple and efficient matter by design, build and integrate data from various sources.

The Data Engineer builds data pipelines from existing data sources and other (on-prem) systems into the Big Data platform; Creates consistent e2e BI solutions with a data lifecycle starting with collecting the needed data until visualizing them; Creates AI/ML models in BigData use cases and helping business to leverage the results; User story creation with good understanding of business processes, product and customer journey; Rates user stories for customer value and to challenge the product owner on prioritization of user stories (e.g., stability versus functionality); User story refinement: Is able to translate user stories into technical stories; Development: Able to design, develop and optimize data storage and data processing solutions and proficient in in one or more program languages like python and SQL to ensure sufficient quality level ,including test criteria; Deployment: Is able to create and execute script based deployments, including rollbacks.

Is able to maintain/setup automated deployment using the VF standard toolset; Operations: Is able to setup automated monitoring and proactive alerting, is able to provide parameters to size the underlying application infrastructure. Do you want to be part of the clean energy transition and make a real impact through data? As a Data Engineer at Vattenfall, you’ll be part of an agile SCRUM team working with advanced analytics tools across various systems to solve complex issues, collaborating closely with Vattenfall Wind stakeholders.

What you’ll do: Your focus will be on designing, building, and maintaining data products that enable real-time and historical wind turbine data sharing with internal and external stakeholders. You will work with Kubernetes, Python, and Message Queuing Technology (MQTT) to create scalable data pipelines and integrate live and high-frequency, and scientific data from our wind turbine fleet. What you bring: You know Python inside out, including DevOps best practices like automated testing, CI/CD and Git. Experience building, deploying, and managing containerized applications using Docker and Kubernetes.

Experienced with Cloud services (preferably Azure). Experience with Databricks and building reliable data pipelines using PySpark, SQL, and Delta Lake. Experience with real-time messaging technologies, such as MQTT or other message queuing systems, for streaming and processing live data. Experience building and consuming APIs for data sharing and integration. As a bonus; familiarity with Apache NiFi or similar data integration and data flow tools. You are familiar with the Agile way of working and methodology. Your command of English is strong.

Good communication skills for explaining complex systems to non-technical stakeholders. You know your strengths and weaknesses. HBO/WO English is the main language we use for communication since the members of our team have different nationalities. We expect the candidates to have a proven trackrecord in the data engineering area The team is present in the Amsterdam office on Tuesdays and Thursdays and we expect the candidate to also be avaiable in Amsterdam on those days. Experience in the energysector is a pre

Project details

• Start: 2027-01-11T23:00:00
• Remote: hybrid
• Location: Amsterdam Noord-Holland, NL
• Languages: English