Senior Data Engineer

Project overview

Join #DiagTeam and help care for the health of millions of people.

At Diagnostyka, we focus on dynamic development. We are constantly expanding our offer – laboratory, imaging, genetic, and histopathological diagnostics. We promote conscious prevention and the idea of longevity.

We are developing DanePlus – a modern data platform based on Google Cloud Platform, which integrates data from multiple systems, supports business analytics, reporting, and the development of artificial intelligence-based solutions.

We are looking for a Senior Data Engineer who will design and develop data solutions in BigQuery, co-create engineering standards, and help us develop the platform towards conversational analytics and AI.

Responsibilities

• Designing, building, and developing data processing processes (ETL/ELT) on the DanePlus platform, from source integration to analytical models and data products.
• Developing a data architecture based on BigQuery and the Medallion approach (Bronze, Silver, Gold), including designing data models supporting business analytics and reporting.
• Integrating data from source systems, particularly Oracle/LIMS, SAP, and operating systems, using batch processing, incremental processing, and CDC.
• Developing and optimizing data transformations and orchestrations using SQL, Python, Dataform, and Cloud Composer.
• Designing analytical models and semantic layers used by Power BI, AI solutions, and conversational analytics.
• Co-creating solutions using LLMs and AI agents for natural language data analysis and exploring new BigQuery capabilities, including AI functions, vector search, and graph analytics.
• Actively using AI-assisted development tools, such as Claude Code, Cursor, or Gemini Code Assist, to automate programming, testing, and documentation of solutions.
• Ensuring the quality, security, and reliability of data processes through automated tests, monitoring, access control, and good engineering practices.
• Optimizing the performance and costs of data processing in BigQuery and other GCP services.
• Co-creating technical standards, CI/CD practices, code review, and Infrastructure as Code.
• Collaborating with data architects, analysts, and data stewards, and supporting the development of the team's technical competencies through mentoring and knowledge sharing.

Requirements

• Minimum 4–5 years of experience in data engineering, including practical experience in building and maintaining cloud solutions, preferably on Google Cloud Platform.
• Very good knowledge of BigQuery and advanced SQL, including data modeling, query optimization, partitioning, clustering, and cost control.
• Practical experience in designing and implementing ETL/ELT processes, including incremental processing and handling large data volumes.
• Good knowledge of Python for data processing, automation, and system integration.
• Knowledge of Data Lakehouse / Medallion architectures and analytical modeling (Kimball, star schema); knowledge of Data Vault will be an additional advantage.
• Experience with data transformation and orchestration tools such as Dataform, dbt, Cloud Composer, or Apache Airflow.
• Practical ability to use AI programming assistants, e.g., Claude Code, Cursor, GitHub Copilot, or Gemini Code Assist, while adhering to data security principles and verifying generated code.
• Knowledge of software engineering best practices: Git, CI/CD, code review, automated tests, and monitoring of production solutions.
• Understanding of data security and access management principles in the cloud, including IAM, RLS, and sensitive data protection.
• Ability to independently design solutions, make technical decisions, and solve complex problems.
• Communication skills, proactivity, and willingness to share knowledge and develop the team's competencies.

Nice to have

We offer:

• Experience with Conversational Analytics, Text-to-SQL, AI agents, and integrating LLM models with BigQuery, e.g., using Gemini or Vertex AI.
• Knowledge of BigQuery Graph / SQL Graph, relationship modeling, and graph analytics.
• Knowledge of AI functions in BigQuery, vector search, embeddings, RAG, and LLM integration with data via MCP.
• Experience with AlloyDB / PostgreSQL and integrating data from relational databases, especially Oracle.
• Knowledge of CDC and streaming technologies, including Debezium, Apache Kafka, Kafka Connect, Strimzi, Pub/Sub, or Dataflow (Apache Beam).
• Experience with SAP data integration, including CDS Views or SAP Datasphere.
• Knowledge of Data Vault 2.0 and designing data models for large analytical environments.
• Experience with Terraform, Azure DevOps, deployment automation, and code testing and quality tools, e.g., pytest and ruff.
• Knowledge of VPC Service Controls, Policy Tags, RLS/CLS, and data anonymization and secure sharing mechanisms.
• Experience with GKE, Cloud Run, or managing data processing services in containerized environments.
• Experience working with medical data or other sensitive data.
• Real impact on the development of the DanePlus platform and the direction of data utilization in one of the largest medical diagnostics organizations in Poland.
• Work with large data volumes and complex technological and analytical challenges.
• Opportunity to develop modern solutions based on Google Cloud, BigQuery, and AI.
• Participation in building internal Data Engineering competencies and technological standards.
• Collaboration with analytical and business teams in creating data products with real applications.
• Full-time employment based on an employment contract or contract.
• Hybrid work (3 days in the office at ul. Prof. M. Życzkowskiego 18).

Project details

• Start: flexible
• Remote: hybrid
• Location: Kraków, Profesora Michała Życzkowskiego 18, PL
• Languages: Polish