Senior Data Engineer

Project overview

Hays IT Contracting is a cooperation based on B2B rules. We connect IT specialists with the most interesting technology projects on the market. For our client, a universal banking institution delivering financial market services and data-driven solutions, we are currently looking for a Specialist for the position of PostgreSQL Data Platform Engineer.

The project focuses on the development and maintenance of a financial market data platform supporting FX analysis processes. The solution operates in a near real-time architecture based on PostgreSQL and Python, processing more than 40 million messages daily across 46 currency pairs. The platform provides data services for teams requiring information refreshed every second, ensuring high availability, scalability, and long-term data retention capabilities. The environment includes integration with messaging systems and support for large-scale data processing workloads.

Technology Environment

Scope of Services

Required Competencies

• Service delivery model: Hybrid, 1 day every 2 weeks from the office in Warsaw
• Rate: 110-148 PLN/h net (+ VAT)
• Cooperation: 12 months with possible extensions
• Cooperation type: B2B, through Hays Poland
• Language: English B2
• PostgreSQL
• SQL
• PL/pgSQL
• Python
• Linux
• RabbitMQ/Kafka
• Apache Airflow
• CI/CD
• Design, develop, and maintain a high-performance financial data platform
• Build and optimize PostgreSQL database solutions for large-scale data processing
• Develop and enhance ETL processes supporting data ingestion and transformation
• Process and manage approximately 40 million messages per day in a near real-time environment
• Support storage and retrieval of data with a two-year retention period
• Optimize database performance through query tuning, execution plan analysis, and index optimization
• Implement and maintain scalable data models aligned with business requirements
• Integrate the platform with RabbitMQ-based communication systems
• Monitor platform performance and contribute to high availability and scalability initiatives
• Support internal stakeholders requiring rapidly refreshed financial market data
• Minimum 5 years of experience working with PostgreSQL
• Strong knowledge of SQL and PL/pgSQL
• Experience in data modeling and ETL processes
• Practical experience in database performance tuning
• Ability to analyze execution plans and optimize indexes
• Experience working with large volumes of data
• Good knowledge of Linux
• Previous experience in the banking or financial sector

Nice to have

Cooperation Process

The cooperation is carried out between independent businesses (B2B model) and covers the performance of services within the agreed project scope. The Contractor independently organizes and manages the manner in which the services are performed, taking into account the project requirements, security obligations, and the necessary communication with the client.

Any information regarding location, availability, or project meetings is provided solely for organizational purposes and shall not be construed as creating an employment relationship, employer-like supervision, or performance of work under the direction or control of the client.

#LI-DNI

#1202164 - Julia Cichocka

• Python programming skills
• Experience with Kafka or RabbitMQ
• Knowledge of Apache Airflow
• Experience with CI/CD practices
• Knowledge of PostgreSQL indexing mechanisms, including B-Tree, Hash, GIN, GiST, and BRIN
• Experience with table partitioning
• Knowledge of materialized views
• Experience with VACUUM, ANALYZE, and database performance monitoring
• Profile and experience review by the Hays team
• A conversation with a Hays consultant about the project and scope of services
• A technical meeting with project representatives (1 stage)
• Agreement on cooperation terms
• Welcome to the project!

Project details

• Start: flexible
• Location: Mazowieckie, PL