Search by job, company or skills

Lead Data Engineer

Lead Data Engineer

Computomics GmbH
8-10 Years
Not Disclosed
Early Applicant
  • Posted 11 days ago
  • Be among the first 10 applicants

Job Description

We're Hiring: Lead Data Engineers ( Immediate to 30 days)

Location: Whitefield, Bangalore

Platform Specialization: Databricks

Project Scope: Complex DWH & Migration initiatives across the full data engineering lifecycle

Are you passionate about building scalable data pipelines, optimizing performance, and solving real-world data challenges We're looking for hands-on Data Engineers who thrive in fast-paced environments and bring deep expertise in SQL, Python, Spark, Databricks, and cloud-native data platforms.

Lead Data Engineers (8+ years):

  • Build and optimize robust data pipelines for ingestion, transformation, and analytics
  • Work across structured and semi-structured data sources, ensuring performance and reliability
  • Collaborate with cross-functional teams to deliver scalable solutions on Databricks
  • Strong coding skills in SQL, Python, and Spark are a must
  • Experience in data validations, reconciliation, and quality checks across ingestion and transformation stages

Must-Have Technical Skills:

  • Advanced SQL (window functions, CTEs, performance tuning)
  • Python (data manipulation, modular design, error handling)
  • Advanced Apache Spark Programming
  • Databricks (Notebooks, Delta Lake, Job Orchestration, Unity Catalog, Delta Live Tables, DB Asset Bundles, Lakehouse Architecture, Workspace Management, Cluster Configuration, MLflow Integration, and Role-Based Access Control)
  • CI/CD pipelines, version control (Git), and testing frameworks
  • Experience with cloud platforms (Azure, AWS, or GCP)

Software Engineering Fundamentals:

  • Clean code practices and modular design
  • Agile delivery and sprint planning
  • Debugging, logging, and exception handling
  • Code reviews and mentoring junior engineers
  • Understanding of system design and scalability

Bonus Skills:

  • Exposure to data modeling (dimensional, star/snowflake schemas)
  • Familiarity with data governance, lineage, and cataloging tools
  • Prior experience in large-scale data migration projects and validation frameworks

More Info

Job Type:
Industry:
Employment Type:

Key Skills

Advanced Apache Spark Programming

Lakehouse Architecture

Role-Based Access Control

Databricks Notebooks

Delta Live Tables

DB Asset Bundles

modular design

CTEs

Job Orchestration

Cluster Configuration

Unity Catalog

CI CD pipelines

Delta Lake

Advanced SQL window functions

MLflow Integration

error handling

About Company

Similar Jobs

10-12 yrs
Bengaluru, India
Skills:
Data Security, Data Modelling, Sql, Nosql, Distributed Systems, Data Privacy, Data Lake, modern data engineering tools and platforms, data pipelines, Big Data architecture, data-centric applications, large-scale data pipelines, data visualisation techniques, Pyspark programming language, distributed storage platforms, data schemas, distributed processing platforms
5-8 yrs
Bengaluru, India
Skills:
BigQuery, Shell scripting, Advanced Sql, Python, IICS Informatica Intelligent Cloud Services, Google Cloud Platform Administration
8-10 yrs
Bengaluru, India
Skills:
snowflake , BigQuery, Spark, Kafka, Databricks, Redshift, Python, Sql, Airflow, dbt
6-9 yrs
Bengaluru, India
Skills:
Bi Launch Pad, Sap Businessobjects, Universe Design, Web Intelligence, Bw On Hana, Data Modeling, Power Bi, SQL Server, Tableau, Sap Bw, Sap Hana, Sap Bo, Oracle, Webi, ETL and Data Warehousing Concepts, Data Analysis, SQL Database Querying, Crystal Reports, Report Performance Tuning, UNX, Reporting, Teradata, SAC, SAP Analytics Cloud, UNV
7-12 yrs
Bengaluru, India
Skills:
Data Modeling, Redshift, Sql, ELT, Jenkins, Python, AWS, Etl, Airflow, SageMaker, GitHub Actions, dbt, Dagster, CI/CD