Location: On-site in Abu Dhabi
We're looking for an experienced, hands-on Data Engineer capable of designing, building, and operating data pipelines and models that power analytics and applications across investment teams and Middle/Back Office of one of our clients.
The ideal candidate has familiarity with financial markets (securities, prices, corporate actions, positions/holdings) and thrives in ambiguous environments - proactively shaping solutions rather than waiting for tickets. You will own data end-to-end: from ingesting vendor and internal sources, to modeling in the lakehouse, to ensuring data is discoverable, reliable, and cost-efficient.
You will collaborate closely with BAs/PMs and quants, anticipate downstream needs, and propose pragmatic architectures that balance speed, governance, and scalability.
Responsibilities
- Participate in requirements clarification and sprint planning sessions
- Design technical solutions and implement them
- Build robust ETL pipelines using PySpark for extraction and transformation
- Optimize and tune existing ETL processes for performance, scalability, and reliability
- Write unit and integration tests
- Support QA teammates during acceptance testing
- Resolve production incidents as a 3rd line engineer
Required Skills
- Minimum 5 years of experience in IT/Data
- Proficiency in PySpark for distributed computing
- Strong Python skills for ETL development
- Strong expertise in writing and optimizing complex SQL queries
- Experience with PostgreSQL, MySQL, Oracle, or Snowflake preferred
- Experience with data warehousing concepts and platforms (ideally Databricks)
- Familiarity with ETL tools and processes
- Experience with Data Modeling
- Proficiency with Git
- Experience with Prometheus, Grafana, or custom monitoring scripts
- Experience implementing data validation, cleansing, and quality frameworks (ideally Monte Carlo)
- Familiarity with Azure DevOps, Jira, and/or Confluence
- Strong debugging and troubleshooting skills for complex data issues
- Strong written and verbal communication skills
- Ability to explain technical concepts to non-technical stakeholders
- Understanding of the Investment Data domain (desired)
