Pyspark

Location: PAN India

Experience: 8 to 10 Years

Key Skills

:PySpark, Pytho

n
Must have Skill

• s:
  Implementing data ingestion pipelines from different types of data sources i.e Databases, S3, Files et
• c..Experience in building ETL/ Data Warehouse transformation proce
• ss.Experience working with structured and unstructured da
• ta.Developing Big Data and non-Big Data cloud-based enterprise solutions in PySpark and SparkSQL and related frameworks/librari
• es,Developing scalable and re-usable, self-service frameworks for data ingestion and processi
• ng,Integrating end to end data pipelines to take data from data source to target data repositories ensuring the quality and consistency of da
• ta,Processing performance analysis and optimizati
• on,Bringing best practices in following areas: Design & Analysis, Automation (Pipelining, IaC), Testing, Monitoring, Documentati

on.
Good to have (Knowled

1. ge):
   Experience in cloud-based solut
2. ions,Knowledge of data management princi