Senior Data Engineer

About the Role

We're seeking a proactive, hands-on Senior Data Engineer to drive our Databricks-centric Lakehouse — Delta tables governed by Unity Catalog, Lakeflow pipelines, and AI/BI layers like Genie and dashboards. The ideal candidate will work autonomously, take full ownership of pipelines end-to-end, and keep pace with the fast-evolving Databricks ecosystem.

Experience: 7–10 years

Type: Independent Contributor (able to operate with minimal supervision)

Location: Abu Dhabi

Email: [Confidential Information]

Key Responsibilities

1. Pipeline Orchestration: Design and build robust data ingestion and orchestration pipelines, with Databricks (Lakeflow Jobs/Workflows) as the primary platform and Azure Data Factory supporting legacy/external needs — taking full ownership of pipeline reliability and the transition toward Databricks-native orchestration
2. Lakehouse Data Engineering: Build and maintain scalable Delta Lake tables under Unity Catalog — schema design, access controls, lineage tracking, and data quality across bronze/silver/gold layers
3. Lakeflow Pipeline Development: Build declarative ETL pipelines using Databricks Lakeflow, incorporating CDC, incremental processing, and data quality validations
4. AI/BI & Genie Enablement: Design and maintain semantic layers, metrics, and certified datasets to support Genie spaces and AI/BI dashboards, enabling natural-language querying and self-service analytics
5. SharePoint/External Source Integration: Configure and maintain Lakeflow Connect (or similar) integrations for ingesting SharePoint and other external data sources into the Lakehouse with minimal manual effort
6. Governance & Best Practices: Apply Unity Catalog governance controls — access policies, data masking, audit logging — and uphold strong engineering practices (CI/CD, version control, testing)

Must-Have Skills

• Strong hands-on Databricks expertise — PySpark/SQL, Workflows/Jobs orchestration, Delta Lake, Unity Catalog
• Working knowledge of Azure Data Factory
• Experience with Lakeflow (Declarative Pipelines/Connect) including CDC and data quality validations
• Exposure to Databricks AI/BI or Genie (semantic layers, certified datasets)
• Familiarity with CI/CD and version control (Git, Databricks Repos/Asset Bundles)
• Demonstrated ability to own deliverables fully and resolve issues independently
• Strong communication skills for working with business/analytics stakeholders

We are an Equal Opportunity Employer committed to fostering a diverse, inclusive, and respectful workplace. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. We believe diversity drives innovation and success, and we welcome individuals from all backgrounds to join our team.