Geforderte Skills
- Google Cloud Platform
- Apache Spark
- Apache Kafka
- Python
- SQL
- PL/SQL
- dbt
- ETL
- Cloudera Hadoop
- Linux
- Bash
- Data Governance
Sprachen: Polish · English
Jobbeschreibung
Seeking an experienced Data Engineer for a project involving the development and maintenance of data processing, with a focus on migrating an Oracle Data Warehouse to Google Cloud Platform (GCP).
Project overview
Be in IT specializes in comprehensive IT specialist recruitment for Polish and international companies. We support clients in acquiring experts in software development, AI, data, cloud, cybersecurity, ERP, CRM, and IT management.
We focus on technological knowledge, clear communication, and a partnership approach, enabling us to effectively match candidates to organizational needs and ongoing projects.
Currently, for our client, we are looking for a person for the position of: Mid GCP Data Engineer.
Responsibilities
We expect:
We offer:
- For a project carried out for our client, we are looking for an experienced person for the Data Engineer position, who will be responsible for the development and maintenance of data processing processes and participation in the migration of the existing Oracle Data Warehouse to the Google Cloud Platform (GCP) environment.
- Design, development, and maintenance of processes for retrieving data from various source systems to Data Lake.
- Building and optimizing complex data pipelines and ETL processes using Apache Spark, Apache Kafka, Python, and SQL.
- Creating and developing frameworks supporting data processing automation.
- Implementation of comprehensive data testing and quality control mechanisms.
- Participation in the migration of the Oracle Data Warehouse to the Google Cloud Platform (GCP) environment.
- Integration of data from various sources, including Kafka, MQ, SFTP, databases, APIs, and file shares.
- Development and optimization of data processing processes using dbt, SQL, and PL/SQL.
- Working with the Cloudera Hadoop ecosystem and the Linux/Bash environment.
- Ensuring solutions comply with Data Governance, security, and data quality principles.
- Collaboration with Data Analysts, Data Scientists, and other technical team members.
- Analysis and implementation of new technologies to improve data processing processes.
- Minimum 3 years of experience working with data in a Data Analyst, Data Quality Analyst, or related role.
- Very good, practical knowledge of GCP (Google Cloud Platform) – a key requirement.
- Experience working with dbt (Data Build Tool).
- Advanced knowledge of SQL and PL/SQL.
- Very good knowledge of Python.
- Practical experience with the Linux system and Bash scripting.
- Knowledge of the Cloudera Hadoop ecosystem, particularly Apache Spark and Apache Kafka.
- Experience in designing and developing ETL processes and real-time data processing.
- Knowledge of Data Quality and Data Governance principles.
- Ability to work with various source systems and data integration methods, including Kafka, MQ, SFTP, API, and databases.
- Higher education in computer science, Data Science, statistics, economics, or related fields.
- Knowledge of English at a minimum B2 level.
- Long-term cooperation with remuneration in the range of 100-130 PLN net/hour + VAT.
- Support in setting up and running a business, for those without such experience.
- Efficient recruitment process
- two technical interviews with managers, online (each up to an hour).
- Fully remote cooperation.
- Benefits
- private medical care, Multisport card.
- Modern equipment provided by the company, along with software and configuration.
Project details
- Start: flexible
- Remote: remote
- Location: Warszawa, Mokotów, PL
- Rate: 100-130 EUR/h
- Languages: Polish, English
Indexiert von der 9am-Meta-Suche. Du bist Job-Anbieter? Listing beanspruchen oder verwalten
Gefunden auf justjoin.it