Job description
Key Responsibilities:
Data Modelling & Engineering:
- Design, implement, and maintain data models using Azure Data Modeling techniques to meet business needs.
- Create and optimize ETL pipelines, leveraging Azure Data Factory and Azure Databricks to process large datasets and integrate data from various sources.
Data Integration:
- Work on integrating data from multiple platforms and databases into a cohesive environment using Azure technologies, ensuring data consistency and quality.
Data Analysis & Reporting:
- Implement data engineering solutions to support reporting, analytics, and business intelligence tasks.
- Utilize Python and SQL to manipulate and analyze datasets to deliver insights and reports.
Cloud Data Services:
- Work with cloud data services within the Windows Azure ecosystem, ensuring efficient data storage, processing, and retrieval.
Collaboration:
- Collaborate with cross-functional teams, including Data Scientists, Analysts, and other Engineers, to support data-driven decision-making processes.
Performance Optimization:
- Ensure high performance and scalability of data models, pipelines, and processes to handle growing data loads efficiently.
Documentation:
- Maintain clear and up-to-date documentation for data engineering processes, models, and pipelines.
Skills and Qualifications:
Hands-on Experience:
- Minimum 5-7 years of experience in Data Engineering or related roles, with proven expertise in Azure Data Modelling, Azure Data Factory, Python, and SQL.
Technical Skills:
- Strong knowledge and hands-on experience with Azure Data Factory, Azure Databricks, and other Azure-based data services.
- Proficiency in SQL for data manipulation, extraction, and reporting.
- Experience with Python for scripting and automating data-related tasks and workflows.
- Knowledge of Windows Azure cloud infrastructure, storage, and computing services.
Data Modelling:
- Experience with Data Modelling techniques, including normalization, schema design, and data integration.
Communication & Collaboration:
- Excellent communication skills with the ability to work collaboratively in a remote team and document solutions clearly.
Problem-Solving:
- Strong analytical and troubleshooting skills to resolve data-related issues and optimize solutions.
