We are seeking a Databricks Architect with deep expertise in Azure Data Platform, Data Modelling, and Medallion (Lakehouse) Architecture. The ideal candidate will design and implement scalable, high-performance data solutions using Databricks, Delta Lake, and modern data engineering practices, enabling enterprise-wide analytics and business intelligence.
Location: Abu Dhabi
Experience: 10–15+ Years
Email: [Confidential Information]
Key Responsibilities
Medallion Architecture & Lakehouse Design
- Design and implement Medallion Architecture (Bronze, Silver, Gold layers) on Azure Databricks
- Define data flow standards across ingestion, cleansing, transformation, and consumption layers
- Ensure data quality, lineage, and governance across all layers
- Optimize storage and compute using Delta Lake features (ACID transactions, time travel, schema evolution)
Architecture & Strategy
- Architect end-to-end Azure data platforms using:
- Azure Databricks
- ADLS Gen2
- Azure Data Factory
- Azure Synapse
- Define data platform standards, best practices, and reusable frameworks
- Lead data modernization initiatives (on-prem to cloud, legacy to lakehouse)
Data Engineering Leadership
- Build scalable pipelines using PySpark, Spark SQL, and Databricks workflows
- Implement batch + real-time ingestion patterns
- Drive performance tuning and cost optimization
- Mentor teams on Spark optimization, partitioning, and job orchestration
Data Modelling
- Design enterprise-grade data models:
- Dimensional (Star/Snowflake)
- Normalized and Data Vault (nice to have)
- Align data models with Gold layer consumption use-cases (BI/Reporting)
- Translate business needs into scalable analytical datasets
Integration & Consumption
- Build ingestion pipelines using:
- ADF, Event Hub, Kafka, APIs
- Enable reporting via:
- Power BI, Tableau, or similar BI tools
- Ensure seamless consumption from Gold datasets
Stakeholder Engagement
- Act as a data architecture SME in client discussions
- Drive solutioning, architecture reviews, and governance forums
- Collaborate with business teams for data-driven decision making
Required Skills
Core
- Strong hands-on experience with Azure Databricks & Delta Lake
- Proven expertise in Medallion Architecture implementation
- Advanced PySpark and Spark SQL
- Experience with ADF, ADLS Gen2, Synapse
Data Modelling
- Strong experience in:
- Dimensional modelling (must-have)
- Data modelling aligned to Gold layer analytics
- Understanding of data quality and governance frameworks
Nice-to-Have
- Real-time processing (Kafka / Spark Streaming)
- Azure Purview or data governance tools
- ML pipelines on Databricks
- Domain: Airlines / BFSI / Retail
Leadership Skills
- Strong architectural thinking
- Client-facing experience
- Ability to lead and scale engineering teams
- Excellent communication and stakeholder management
We are an Equal Opportunity Employer committed to fostering a diverse, inclusive, and respectful workplace. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. We believe diversity drives innovation and success, and we welcome individuals from all backgrounds to join our team.
