We are seeking a highly skilled and experienced Azure Data Engineer to join our team. The ideal candidate will have a strong background in cloud-based data integration, data transformation, and analytics solutions, with a focus on Azure services. This role involves designing, implementing, and maintaining robust data pipelines and analytics frameworks to support business intelligence and advanced analytics initiatives.

Key Responsibilities:

Data Engineering and Development:

Design and implement scalable and efficient ETL/ELT processes using Azure Data Factory and Azure Databricks (PySpark).
Manage and optimize data storage and access in Azure Data Lake Storage.

Data Architecture:

Develop and maintain a Lakehouse architecture for data analytics and reporting.
Ensure best practices in data organization, partitioning, and access control in Azure Data Lake.
Collaborate on the design and implementation of Microsoft Fabric architecture to enhance data accessibility and governance.

Data Analysis and Reporting:

Create and optimize reports and dashboards using Power BI to enable data-driven decision-making.
Develop T-SQL scripts and stored procedures for data transformation and analysis.

Cloud Infrastructure and DevOps:

Leverage knowledge of Azure IaaS services for data platform setup and maintenance.
Implement and manage CI/CD pipelines using Azure DevOps to automate data pipeline deployments.

Technical Expertise and Collaboration:

Provide strong technical guidance in Spark architecture and ensure best practices for PySpark code development.
Collaborate with cross-functional teams to align on data strategies and solutions.
Troubleshoot and resolve complex data engineering challenges efficiently.

Mandatory Skills:

Proficiency in Azure Data Factory, Azure Data Lake Storage, and Azure Databricks (PySpark).
Strong understanding of ETL and ELT principles.
Deep knowledge of Lakehouse architecture and its implementation.
Expertise in PySpark and Spark architecture.
Solid understanding of Azure Data Lake architecture and access control mechanisms
Strong command of T-SQL for advanced querying and data manipulation..

Good to Have:

Experience with Power BI for visualization and reporting.
Familiarity with Microsoft Fabric architecture.
Knowledge of Azure IaaS services.
Understanding of CI/CD processes and tools like Azure DevOps.

Qualifications:

Bachelors degree in Computer Science, Information Technology, or a related field.
5+ years of experience in data engineering with a focus on Azure services.
Proven expertise in designing and implementing data pipelines and architectures in cloud environments.
Strong problem-solving skills and ability to work collaboratively in a team.

Preferred Certifications:

Microsoft Certified: Azure Data Engineer Associate.
Microsoft Certified: Azure Solutions Architect Expert.

Microsoft Certified: Power BI Data Analyst Associate

Azure Data Engineer/Architect