Job Description
We are seeking a highly skilled Senior Data Engineer to join our data and analytics team. The ideal candidate will have extensive experience in working with diverse data sources, performing data profiling, and identifying matching records across systems to create master data records. This role requires strong analytical and technical skills, along with expertise in data quality, integration, transformation, and cloud-based data platforms.
Roles & Responsibilities
Data Source Analysis:
- Analyze multiple structured and unstructured data sources to understand schemas, data quality, and relationships
- Assess and document the metadata, formats, and business rules of incoming data sets
Data Profiling & Matching:
- Perform comprehensive data profiling to assess completeness, accuracy, uniqueness, and consistency of data
- Develop logic to match and reconcile records across different systems using keys, rules, or fuzzy matching techniques
Master Data Management (MDM):
- Design and implement processes for building and maintaining golden master records
- Support creation and maintenance of master data across domains such as customer, product, asset, etc
- Handle data deduplication and standardization
ETL & Data Pipelines:
- Build scalable data pipelines using Azure Data Factory for ingesting, transforming, and cleansing data from multiple sources
- Ensure efficient data movement across staging, processing, and analytical layers
- Leverage Azure Synapse Analytics (optional) for advanced data processing and analytics
Collaboration & Governance:
- Work closely with business analysts, data stewards, and application owners to understand data definitions and ownership
- Enforce data governance standards and best practices
Performance & Optimization:
- Monitor data workflows and optimize performance for large-scale processing
- Troubleshoot data issues and implement robust logging and monitoring
Requirements
Requirements
Qualification and Education Requirements:
- Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field
Minimum Experience/Skills Required:
- 5+ years of experience in data engineering, ETL development, or data integration
- Proven experience in data profiling, matching, and MDM (Master Data Management)
- Expertise in SQL for querying, transforming, and optimizing complex datasets
- Strong proficiency in Python or PySpark for data processing and pipeline development
- Experience with Azure Data Factory for building and managing data pipelines
- Familiarity with Azure Synapse Analytics is a plus
- Strong expertise in ETL/ELT tools (e.g., Talend, Informatica, SSIS, Apache NiFi)
- Familiarity with data quality frameworks and metadata management
- Strong communication skills in both Arabic and English
Preferred Skills
- Experience with data cataloging tools (e.g., Azure Purview, Alation)
- Knowledge of data modeling and normalization techniques
- Understanding of data privacy and compliance standards (GDPR, HIPAA, etc.)
- Familiarity with REST APIs and integrating with third-party data services
