Looking for a candidate with 7+ to 10 years of hands-on experience in Data Engineering with Azure/ Microsoft Fabric. The candidate should have good knowledge on Market data.
- Experience in financial services, particularly with market data platforms, vendor data ingestion (Bloomberg, MSCI, LSEG, ICE), and entitlement management.
- Design and implement a Medallion Architecture (Landing/Bronze, Conformed/Silver, Data Products/Gold) with delta tables using either Azure services (ADLS, Azure Databricks, Azure Synapse Analytics) or Microsoft Fabric Lakehouse.
- Implement SCD Type 2 patterns for slowly changing dimension tracking across conformed and product layers.
- Contribute to the design and integration of a Data Quality (DQ) Studio framework, including push-down execution, rule catalogs (Technical, Business, Custom Business, Monitoring), and API-first resilience patterns.
- Support BCDR (Business Continuity & Disaster Recovery) strategies, in either Azure (Databricks, Azure Synapse Analytics, Azure SQL, Azure CosmosDB) or in Fabric (workspace replication patterns for Fabric Warehouse, Fabric SQL Database via SqlPackage, and Mirrored Cosmos DB.)
- Collaborate with Stakeholders and gather the requirements.
- Design and implement data pipelines in Azure or Microsoft Fabric.
- Ingest data from multiple data sources using Azure or Microsoft Fabric.
- Work on data transformations using PySpark/ Spark SQL/ T-SQL
- Create semantic models and create power BI reports.
- Create CI/CD deployments
- Troubleshoot performance issues.
- Create Fabric Data Agents.
- Create OneLake shortcuts in Microsoft Fabric.
