Senior Data Engineer

Job Title: Senior Data Engineer

Location: UAE (Onsite)

Experience: 10+ Years (Mandatory)

Notice Period: Maximum 30 Days (Mandatory)

Job Summary

We are looking for highly experienced Senior Data Engineers with 10+ years of relevant experience in designing and building enterprise-scale data engineering solutions. The ideal candidate will have strong expertise in Microsoft Azure, Cloudera, Hadoop ecosystem, Spark, and modern data platforms, with hands-on experience in developing scalable data pipelines for analytics, business intelligence, and machine learning solutions.

Key Responsibilities

Design, develop, and maintain scalable data pipelines for enterprise Data Lake, Data Warehouse, and Analytics platforms.

Build robust ETL/ELT pipelines to ingest, transform, and model data from multiple sources.

Translate functional and non-functional requirements into scalable technical solutions.

Develop high-performance data engineering solutions while ensuring security, scalability, and reliability.

Debug complex production issues, troubleshoot data pipeline failures, and resolve incidents efficiently.

Perform code reviews and ensure adherence to coding standards, best practices, and governance policies.

Collaborate with architects, data scientists, BI developers, and cross-functional teams to deliver business solutions.

Develop dimensional data models for reporting and analytics.

Maintain metadata, data lineage, and data governance documentation.

Perform unit testing, defect resolution, and production support activities.

Participate in architecture discussions and contribute to continuous platform improvements.

Work within Agile and Waterfall delivery methodologies.

Mandatory Technical Skills

Microsoft Azure Data Platform

Cloudera Technology Stack

Hadoop Ecosystem

Apache Spark

Scala

Hive

HBase

Sqoop

Oozie

Apache NiFi

Apache Airflow

HDFS

Azure Data Lake Storage (ADLS Gen2)

Azure Data Factory (ADF)

Azure Databricks

Kafka

Elasticsearch

AVRO & PARQUET file formats

Spark Streaming

SnapLogic

TIBCO

Snowflake

SQL

MicroStrategy

Power BI

Git / Bitbucket

Azure DevOps

Required Experience

Minimum 10+ years of overall IT experience.

Minimum 8+ years of hands-on, end-to-end Data Engineering experience.

Proven experience building enterprise-scale data pipelines and modern analytics platforms.

Strong experience in Azure and Cloudera-based data engineering environments.

Experience working with structured and unstructured data at scale.

Hands-on experience with data ingestion, transformation, modelling, and dimensional data warehouse design.

Strong knowledge of data governance, metadata management, and data lineage.

Experience supporting production environments and resolving complex technical issues.

Familiarity with CI/CD pipelines and DevOps practices.