Project Description:
L2 DataOps Consultant - Lakehouse Support. The consultant provides operational diagnostics and incident management for Lakehouse data platforms in an investment management environment . The role owns L2 incident resolution across data pipelines, coordinates with platform engineering, Data Engineers and data domain teams, and applies structured ITSM, change and problem management to reduce recurring issues. Working knowledge of investment-management data domains (portfolio/positions, trade lifecycle, market data, risk & performance) is required for correct incident interpretation and faster RCA. Engagement: onsite, 6 months, 3 resources, start ASAP.
Responsibilities:
- Own and manage L2 incident resolution for Lakehouse data platforms
- Perform deep operational diagnostics across data pipelines (batch/stream)
- Coordinate resolution across platform engineering, Data Engineers and data domain teams
- Ensure correct usage of ITSM tools (e.g., ServiceNow) for incident tracking and lifecycle management
- Leverage metadata and documentation for troubleshooting and root cause analysis
- Maintain clear ownership, escalation paths and communication during incidents
- Reduce incident noise and recurring issues through structured problem management
Mandatory Skills Description:
- Must have - Data Engineering skills: work as Data Engineer, work with DeltaLake, Databricks environment. Proven hands-on experience working with Delta Lake under the hood
- Also,
- Strong experience in Data Operations (L2 support)
- Solid understanding of Lakehouse architectures (Databricks, Delta Lake or equivalent)
- Hands-on: incident management (ITSM discipline), data pipeline troubleshooting (batch/stream), cross-platform diagnostics
- Working knowledge of metadata-driven operations, data lineage and observability concepts
- Proven ability to own incidents end-to-end; strong RCA, change and problem management
- Basic-to-intermediate knowledge of investment-management data: portfolio/positions, trade lifecycle, market data, risk & performance
- Preferred: enterprise-scale data platforms, observability tools (e.g., Dynatrace), batch orchestration tools
Nice-to-Have Skills Description:
- Palantir (Foundry) — DQC setup in workshop, diagnostics of build failures and data pipelines
- Monte Carlo — data quality alert management and observability
- Azure DevOps — repository management, CI/CD
- Working knowledge of investment management data: portfolio terminology (maturity date, value date), trade lifecycle, market and risk data; familiarity with domain systems (Optima, Cardia)
- Understanding of data lineage, observability and metadata-driven operations concepts
- Exposure to enterprise-scale data platforms and batch orchestration / data workflow tools
