About the role
Job Summary
We are looking for an experienced Azure Data Engineer to design, develop, and maintain scalable data pipelines and cloud-based data solutions using Microsoft Azure. The ideal candidate will have strong expertise in Azure Data Factory, Azure Databricks, PySpark, SQL, and data warehousing concepts.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory.
- Build and optimize data processing workflows using Azure Databricks and PySpark.
- Develop complex SQL queries, stored procedures, and data transformation logic.
- Integrate data from multiple sources, including databases, APIs, files, and cloud storage.
- Implement data lake and lakehouse solutions using Azure Data Lake Storage Gen2.
- Work with Delta Lake and implement efficient data processing and storage strategies.
- Ensure data quality, accuracy, security, and reliability across data pipelines.
- Optimize data pipelines and SQL queries for performance and cost efficiency.