We are seeking a highly skilled Senior Data Engineer with 7 to 10 years of experience to join our team in Bangalore. This is a fully work-from-office role in a fast-paced, consulting-oriented environment. You will design, build, and optimize scalable data pipelines, ensuring high data quality and seamless cross-regional collaboration. The ideal candidate pairs deep technical expertise in Databricks and Azure with a strong problem-solving mindset.
Key Responsibilities
- Pipeline Development: Design and implement scalable data ingestion frameworks handling batch, Change Data Capture (CDC), and streaming data.
- Databricks Architecture: Build and manage robust data solutions using Databricks Delta Lake, notebooks, jobs, and automated workflows.
- Azure Engineering: Orchestrate complex data flows across Azure Data Factory (ADF), ADLS Gen2, and Event Hub.
- Data Quality: Implementation of comprehensive data quality rules, validation logic, and performance optimization protocols.
- Global Collaboration: Partner with teammates across regions—sharing context early, documenting architectural decisions, and maintaining strong handoffs.
- DevOps Integration: Manage end-to-end CI/CD deployment pipelines using Azure DevOps.
Requirements & Qualifications
- Experience: 7 to 10 years of professional experience in Data Engineering.
- Location: Must be willing to work fully from our Bangalore office.
- Core Technical Skills: Advanced proficiency in Python, PySpark, and SQL.
- Platform Expertise: Hands-on experience with Databricks (Delta Lake) and Azure Data Engineering ecosystems.
- Agility: Comfort working in a fast-paced, consulting-oriented environment where priorities shift and outcomes matter.
- Mindset: A strong problem-solving approach with a deep willingness to learn how development, quality testing, bugs, and resolutions fit into overall system design.
- Teamwork: Proven ability to work independently while seamlessly collaborating and sharing concepts with teammates.
