Search by job, company or skills

Lead Data Engineer

Lead Data Engineer

Recro
4-10 Years
Not Disclosed
Early Applicant
  • Posted 9 days ago
  • Be among the first 40 applicants

Job Description

. Job Role : Senior / Lead Data Engineer

• Experience Level: 4 – 10 Years

• Employment Type: Full-time

• Location: Hybrid / Remote / Onsite

• Role Purpose

• As a Senior Data Engineer, you will own the end-to-end data pipelines that feed our analytics and production machine learning models. You are responsible for ensuring that massive volumes of raw data arrive reliably, scale gracefully, and can be trusted across the entire enterprise. • Key Responsibilities

• Architect, implement, and maintain scalable batch and real-time streaming data pipelines using Apache Spark, Kafka, and Python/Scala.

• Design and manage cloud data warehouse and lakehouse architectures (Snowflake, Databricks, BigQuery).

• Implement robust pipeline orchestration using Apache Airflow or Prefect.

• Enforce data governance, quality frameworks, schema evolution, and automated unit/integration testing on data assets.

• Partner with Data Scientists and ML Engineers to supply reliable, low-latency features and ingestion workflows.

• Qualifications & Requirements

• Relevant Experience: 4–10 years of dedicated software engineering and data pipeline development experience.

• Core Technical Stack: Advanced Python and/or Scala, expert SQL, Spark (PySpark), Kafka/Kinesis, Airflow, and Snowflake/Databricks.

• Proven Track Record: Tangible history of processing multi-terabyte/petabyte workloads and handling millions of events/day in production.

• Engineering Standards: Deep understanding of containerization (Docker), version control (Git), and CI/CD for data pipelines.

More Info

Job Type:
Industry:
Employment Type:

About Company

Similar Jobs

10-12 yrs
Bengaluru, India
Skills:
Data Security, Data Modelling, Sql, Nosql, Distributed Systems, Data Privacy, Data Lake, modern data engineering tools and platforms, data pipelines, Big Data architecture, data-centric applications, large-scale data pipelines, data visualisation techniques, Pyspark programming language, distributed storage platforms, data schemas, distributed processing platforms
5-8 yrs
Bengaluru, India
Skills:
BigQuery, Shell scripting, Advanced Sql, Python, IICS Informatica Intelligent Cloud Services, Google Cloud Platform Administration
8-10 yrs
Bengaluru, India
Skills:
snowflake , BigQuery, Spark, Kafka, Databricks, Redshift, Python, Sql, Airflow, dbt
6-9 yrs
Bengaluru, India
Skills:
Bi Launch Pad, Sap Businessobjects, Universe Design, Web Intelligence, Bw On Hana, Data Modeling, Power Bi, SQL Server, Tableau, Sap Bw, Sap Hana, Sap Bo, Oracle, Webi, ETL and Data Warehousing Concepts, Data Analysis, SQL Database Querying, Crystal Reports, Report Performance Tuning, UNX, Reporting, Teradata, SAC, SAP Analytics Cloud, UNV
7-12 yrs
Bengaluru, India
Skills:
Data Modeling, Redshift, Sql, ELT, Jenkins, Python, AWS, Etl, Airflow, SageMaker, GitHub Actions, dbt, Dagster, CI/CD