About The Client: Client is a pioneering Software Engineering and IT consultancy company, transforming and executing at the intersection of Domain and Technology to create digital leaders for our people, clients, partners, and communities.
About The Job:
This role builds reliable, scalable, and governed data pipelines that power analytics, operational reporting, and the Data Intelligence Layer on Databricks.
Essential Job Functions:Primary Responsibilities
Build batch pipelines using Delta Lake with correct table design, partitioning strategy, OPTIMIZE, Z-ORDER, and VACUUM management.
Implement incremental ingestion using Databricks Autoloader with schema evolution controls and checkpointing.
Build Structured Streaming pipelines with watermarking, state handling, late-arriving data logic, and restart safety.
Implement Lakeflow pipelines for declarative, governed data processing.
Design pipelines to support idempotency, replayability, and safe backfills.
Apply Databricks Runtime and Spark optimisations including adaptive query execution, skew mitigation, shuffle tuning, and join optimisation.
Build curated and analytics-ready datasets consumed by Databricks SQL, Genie, dashboards, and downstream applications.
Develop DBSQL views and models aligned with semantic consistency required by the Data Intelligence Layer.
Package pipelines using Databricks Repos and Asset Bundles for CI/CD-driven deployments.
Instrument pipelines with operational metrics and observability hooks.
Secondary Responsibilities
Implement pipelines in compliance with Unity Catalog permissions and data contracts.
Embed data quality checks defined by governance standards.
Support Synapse to Databricks migrations including logic rewrite, validation, and performance benchmarking.
Collaborate with DataOps to ensure deployment readiness and operational stability
Qualifications:
Experience: 5+ Years
How to Apply: Interested candidates are encouraged to respond/submit their updated resumes, and for additional job opportunities, please visit Jobs In India - VARITE.
Unlock Rewards: Refer Candidates and Earn. If you're not available or interested in this opportunity, please pass this along to anyone in your network who might be a good fit and interested in our open positions. VARITE offers a Candidate Referral program, where you'll receive a one-time referral bonus based on the following scale if the referred candidate completes a three-month assignment with VARITE.
Experience Level Bonus Referral:0-2 yearsINR 5,0002-6 yearsINR 7,5006+ yearsINR 10,000 About VARITE: VARITE is a global staffing and IT consulting company providing technical consulting and team augmentation services to Fortune 500 Companies in USA, UK, CANADA and INDIA. VARITE is currently a primary and direct vendor to the leading corporations in the verticals of Networking, Cloud Infrastructure, Hardware and Software, Digital Marketing and Media Solutions, Clinical Diagnostics, Utilities, Gaming and Entertainment, and Financial Services.
Equal Opportunity Employer: VARITE is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. We do not discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, marital status, veteran status, or disability status.
