Lead Data Operations Engineer

Manage and monitor ETL/ELT pipelines, analytics reporting jobs, ML workflows across Azure services, ensuring SLA compliance and the timely resolution of Incidents, failures and performance bottlenecks.

·       Proactively monitor platform health, performance, and SLAs using Databricks monitoring, logging and alerting, to minimize downtime and data issues.

·       Perform root cause analysis and implement long term fixes that prevent recurring issues and improve overall operational reliability.

·       Partner with data engineers, ML engineers, and BI engineers to troubleshoot issues, implement improvements, automate operations, and drive continuous enhancement of data/ML platform reliability.

·       Contribute to platform automation, CI/CD for data/ML pipelines.

·       Provide support for tools that are owned by Enterprise Data Management.

·       Develop and maintain code enhancements, automation scripts that support both data engineering and operational needs.

·       Provide after-hours support for critical systems and applications.

·       Develop and maintain documentation, runbooks, and operational standards.

·       Strong experience in SQL, Python and PySpark

·       Extensive hands-on experience with Azure Databricks with Unity Catalog, ADF, ADLS

·       Excellent problem-solving skills and attention to detail.

·       Strong communication and collaboration abilities across technical and non-technical teams.