Python, PySpark, ETL Developer

2-5 Years

Posted 16 days ago
Over 50 applicants have applied

Job Description

Technology->Analytics - Packages->Python - Big Data,Technology->Big Data - Data Processing->PySpark, ETL

Data Pipeline Development

Develop and maintain scalable batch ETL pipelines using Python and PySpark for data ingestion, transformation, and loading.
Implement reusable transformation logic, ensuring pipelines are modular, testable, and easy to maintain.
Optimize Spark jobs for performance (partitioning, caching, joins, shuffles) and cost efficiency. Data Quality & Reliability
Apply data validation checks, handle schema evolution, and ensure accuracy and completeness of processed datasets.
Troubleshoot pipeline failures, analyze logs, and implement robust error handling and retry mechanisms.
Monitor job runs and support operational stability through alerts, runbooks, and timely incident resolution. Collaboration & Delivery
Work with cross-functional teams to gather requirements, define data mappings, and deliver datasets aligned to business needs.
Participate in code reviews, follow engineering best practices, and contribute to continuous improvement of standards and tooling.
Document pipeline logic, dependencies, and operational procedures for smooth handovers and long-term maintainability.
Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field (or equivalent practical experience).
2–5 years of hands-on experience building data pipelines using Python and PySpark.
Strong understanding of ETL concepts, data transformations, and handling large-scale datasets.
Proficiency in writing clean, maintainable code and debugging production issues.
Working knowledge of data structures, algorithms, and software development best practices.

More Info

Job Type:

Industry:

Function:

Employment Type:

About Company

Infosys

Job ID: 151435127

Jobs by Skill - IT

Jobs by Skill - Non IT

International Jobs

Last Updated: 21-07-2026 09:06:52 PM

Homejobs in Hyderabad / Secunderabad, TelanganaPython, PySpark, ETL Developer

Similar Jobs

GCP Data Engineer_Pyspark/Scala

zorba ai

5-7 yrs

Hyderabad, India

Skills:

BigQuery, Google Cloud Platform, Data Warehousing, Sql, ELT, Apache Airflow, Cloud Storage, DataFlow, Python, Etl, Pub Sub, Cloud Composer

Senior Full Stack Developer (React, Python)

accellor

5-7 yrs

Hyderabad, India

Skills:

react.js , Node.js, Python

Lead Software Engineer (Python Developer)

The Depository Trust & clearing Corporation

4-8 yrs

Hyderabad, India

Skills:

Automation Scripting, Git, Database Technologies, Python development, Rest Apis, Sql, Data Processing, workflow automation, CI CD pipelines, API integrations

Python Developer

Synechron

6-8 yrs

Hyderabad, India

Skills:

Machine Learning, Microservices, Nosql, Nlp, Docker, Flask, Python, Sql, Django, Git, FastAPI, Rest Apis, Kubernetes, vector databases, Pinecone, Claude, Azure OpenAI, RAG Retrieval-Augmented Generation, ChromaDB, Generative AI, LLM-based solutions, FAISS, CI CD pipelines, OpenAI, Weaviate

Python Full Stack Developer with React.js (6-10 Years), Bangalore/ Chennai/ Hyderabad

Tata Consultancy Services

6-10 yrs

Hyderabad, India

Skills:

react.js , Cloudformation, PostgreSQL, Redux, Redis, Django, Git, Typescript, Docker, Terraform, MySQL, Flask, MongoDB, FastAPI, Azure, Kubernetes, Jest, Python, Pytest, AWS, React Testing Library, GitHub Copilot, GitHub Actions