Big Data Engineer

Smartera 3S Solutions and Systems

Egypt, Cairo

2-4 Years

Save

Posted 16 hours ago
Be among the first 10 applicants

Early Applicant

Job Description

Big Data Engineer that designs, builds, and optimizes data pipelines and workflows on Cloudera Data Platform. The role focuses on enabling secure, high-performance data processing to support analytics, regulatory reporting, and data governance across the bank.

Key Responsibilities

Design and implement scalable data ingestion and transformation pipelines using Spark, Hive, Kudu, NiFi, and Kafka.
Build and manage data flows in Apache NiFi to automate ingestion from diverse banking systems and external sources.
Optimize data storage and query performance on HDFS, Impala, and Hive for regulatory and analytical workloads.
Develop ETL/ELT processes that ensure data quality, lineage, and consistency across the data lake and warehouse.
Integrate various banking-related data from core banking, risk, and compliance systems into the enterprise data platform.
Collaborate with Data Scientists, Analysts, and Platform Administrators to deliver reliable, governed datasets.
Apply banking data security and compliance standards (Ranger, Atlas, Kerberos, encryption) to all workflows.
Automate workflows and orchestrate jobs using Airflow, Oozie, or NiFi Flow Orchestration capabilities.

Requirements

Bachelor's degree in Computer Science, Information Systems, or related field.
2+ years of experience in big data engineering using the Cloudera/Hadoop ecosystem.
Strong programming skills in Python, SQL, and Spark (PySpark or Scala).
Hands-on experience with NiFi, Kafka, Hive, Impala, and data integration frameworks.
Familiarity with financial data domains and regulatory requirements.
Solid understanding of data modeling, governance, and performance tuning.
Cloudera Certified Data Engineer or equivalent certification preferred.