Geforderte Skills
- SQL
- Python
- PySpark
- Apache Spark
- Databricks
- Azure Data Factory
- Apache Kafka
- ETL/ELT
- Data Lakehouse
- Git
- CI/CD
- Data Governance
Sprachen: English
Jobbeschreibung
Project overview
Job Summary
Job Description: Senior Data Engineer
Experience: 10+ Years
Role: Senior Data Engineer / Lead Data Engineer
Employment Type: Full-Time
Role Overview
We are looking for an experienced Senior Data Engineer with 10+ years of experience in designing, developing, and supporting enterprise-scale data platforms. The candidate should have strong expertise in data engineering, ETL/ELT, cloud technologies, data warehousing, and distributed data processing, with the ability to provide technical leadership and deliver scalable, reliable data solutions.
Key Responsibilities
Required Technical Skills
Skill Requirements
Other Requirements
- Design and develop scalable data pipelines, ETL/ELT workflows, and data integration solutions.
- Build and optimize enterprise data warehouses, data lakes, and lakehouse architectures.
- Develop complex data solutions using Python, SQL, Spark/PySpark, and related technologies.
- Work with cloud platforms such as Microsoft Azure, AWS, or Google Cloud Platform.
- Implement batch and real-time data processing solutions using technologies such as Apache Spark, Kafka, Databricks, Azure Data Factory, or equivalent tools.
- Perform data modeling, transformation, cleansing, validation, and performance optimization.
- Ensure data pipelines meet requirements for availability, scalability, security, data quality, and performance.
- Troubleshoot production data issues and perform root-cause analysis and permanent corrective actions.
- Implement monitoring, logging, alerting, and automated recovery for critical data pipelines.
- Collaborate with architects, application teams, analytics teams, business stakeholders, and infrastructure teams.
- Participate in architecture discussions, technical reviews, estimation, and solution design.
- Establish coding standards and engineering best practices and conduct code/design reviews.
- Mentor junior engineers and provide technical guidance to the team.
- Support CI/CD and DevOps practices for data engineering workloads.
- 10+ years of overall IT/data engineering experience.
- Advanced SQL skills, including query optimization and performance tuning.
- Strong programming experience with Python and PySpark/Spark.
- Strong understanding of ETL/ELT architecture and data integration patterns.
- Experience with relational and analytical databases such as SQL Server, Oracle, PostgreSQL, Snowflake, or similar.
- Strong experience with at least one major cloud platform: Azure, AWS, or GCP.
- Hands-on experience with data engineering technologies such as:
- Databricks / Apache Spark
- Azure Data Factory / AWS Glue / equivalent
- Kafka or other streaming technologies
- Data Lake / Lakehouse platforms
- Experience with Git, CI/CD pipelines, automated deployments, and version control.
- Knowledge of data governance, security, data quality, lineage, and access-control principles.
- Strong understanding of scalable and high-availability data architectures.
Project details
- Start: flexible
- Location: TN - Tennessee, US
- Languages: English
Indexiert von der 9am-Meta-Suche. Du bist Job-Anbieter? Listing beanspruchen oder verwalten
Gefunden auf hrmsportal.app