Data Engineer

Project overview

VENDORS OR C_2_C CANDIDATES WILL NOT BE CONSIDERED!!!

Data Engineer

Key Responsibilities

Design, develop, and maintain scalable ETL/ELT pipelines using Informatica, Azure Data Factory and Databricks.

Process and transform structured and unstructured data from various sources into a centralized data lake or warehouse.

Leverage Azure Data Lake Storage and Databricks Lakehouse to build efficient data storage and analytics solutions.

Ensure seamless integration with Azure services such as Azure Event Hubs, Azure Stream Analytics, and Azure Functions.

Optimize data pipelines for performance, scalability, and cost-effectiveness.

Implement partitioning, indexing, and caching strategies to handle large-scale datasets.

Key Skills :

5 years of experience as a data engineer or in a similar role.

Hands-on expertise with Informatica, Azure Data Factory, Azure Synapse, and Azure Data Lake.

Proficiency in Databricks for data transformation, analytics, and pipeline orchestration.

Strong programming skills in Python, Scala, or SQL.

Experience with distributed data processing frameworks (e.g., Apache Spark).

Familiarity with CI/CD pipelines and version control systems (e.g., Azure DevOps, Git).

Experience with Delta Lake and Databricks.

Knowledge of real-time data ingestion and processing.

Exposure to BI tools such as Power BI or Tableau.

Strong problem-solving and debugging capabilities.

Excellent communication and collaboration skills.

Knowledge of data modeling techniques and schema designs for analytics.

Preferred Azure / Databricks certifications.

\

Job Facts: Contract W2 | Contract Independent | Remote | $0 - $0

Skills: Microsoft Power BI, Data Lake, Data Modeling, Data Processing, Data Storage, Databricks, Communication, Conflict Resolution, Continuous Delivery, Continuous Integration, Analytics, Apache Spark, Business Intelligence, Git, Informatica, Caching, Collaboration, Problem Solving, Python, Real-time, SQL, Scala, Debugging, DevOps, ELT, Extract, Transform, Load, Microsoft Azure, Orchestration, Scalability, Storage, Tableau, Unstructured Data, Version Control, Warehouse

Reference: Dice Id: ylcon | Position Id: 9135852

Project details

• Start: flexible
• Remote: remote
• Location: US