Geforderte Skills
Jobbeschreibung
Project overview
Our client is seeking an experienced Snowflake & dbt Data Engineer to lead the ingestion, modelling, and architectural optimization of massive data streams for our client in Lisbon. You will be responsible for handling high-volume third-party (eg, AppsFlyer) and first-party data sources, processing 4-5 TB daily.
In this hands-on contract role, you will design robust direct connections, optimize variant loading, and build scalable, cost-effective ELT pipelines using dbt medallion architecture to supply production-ready data marts for downstream analytics.
Key Responsibilities
Required Skills & Qualifications
- Scalable Pipeline Architecture: Design direct connections and build cost-effective ELT pipelines following the dbt medallion architecture (Bronze/Silver/Gold).
- High-Volume Ingestion: Optimize raw JSON variant data loading into Snowflake, managing daily volumes of 4-5 TB seamlessly.
- Performance Tuning: Implement incremental models, advanced table clustering, and performance optimizations to manage query costs and compute efficiency.
- Data Integrity & Operations: Deploy schema drift protection, automated validation routines using zero-copy cloning, and proactive monitoring to ensure analytics-ready data availability.
- Snowflake Expertise: Extensive experience managing high-throughput data warehouses, raw JSON variant processing, table clustering, and zero-copy cloning.
- dbt Mastery: Deep practical knowledge of building production-grade dbt models, medallion architectures, and incremental loading strategies.
- Big Data Processing: Proven track record handling multi-terabyte daily datasets (4+ TB) from both first-party streams and third-party platforms (eg, AppsFlyer).
- Data Modeling: Strong foundation in dimensional modelling, schema management, and schema drift mitigation.
Project details
- Start: flexible
- Duration: 6 months
- Remote: onsite
- Location: Lisboa, Portugal, PT
Indexiert von der 9am-Meta-Suche. Du bist Job-Anbieter? Listing beanspruchen oder verwalten
Gefunden auf FreelancerMap COM