Pyspark Developer
- Posted a month ago
- Over 50 applicants have applied
Job Description
- Develop and optimize scalable data pipelines using Databricks and Apache Spark.
- Design and implement solutions for data integration, data processing, and data storage on Databricks.
- Collaborate with data scientists and analysts to build end-to-end analytics solutions.
- Build and maintain automated workflows, ensuring reliability and efficiency.
- Design and manage ETL processes in Databricks.
- Write optimized Spark SQL and PySpark code for data transformations.
- Work with large datasets and ensure performance, scalability, and reliability of data pipelines.
- Monitor and troubleshoot existing data pipelines and workflows.
- Integrate Databricks with other tools and systems like AWS, Azure, or Google Cloud.
Role: Data Engineer
Industry Type: IT Services & Consulting
Department: Engineering - Software & QA
Employment Type: Full Time, Permanent
Role Category: Software Development
Education
UG: Any Graduate
More Info
Job Type:
Industry:
Function:
Employment Type:
Key Skills
About Company
Innova Solutions is a trusted global technology partner specializing in transformative digital solutions. Innova's robust technology and talent offerings span across an array of industries and include generative AI, data & analytics, cloud computing, digital product engineering, and cybersecurity. Through offices and global delivery centers across North America, Europe, and APAC, our Innovators deliver strategic technology and business transformation solutions to clients enabling them to operate as leaders within their fields.
Skills:
Pyspark, Cloud Storage, Git, Gcp, Postgres, Dataproc, Sql, Python, Big Query, Airflow, BigQuery SQL
Skills:
data wrangling , Unit Testing, Concurrency, Data Pipeline, Python, Pyspark, Apis, Debugging, Oop, Git, Numpy, Pandas, Exception Handling, Data Loading, Restful Services, Decorators, Dataframes, Parallel Processing, Lambda Functions, Error Handling, Vector Operations, Applymap, Apply Map Functions, Gen Agentic AI, DB Connectivity
Skills:
AWS, Sql, Python, Azure, Gcp, Scala, Pyspark
Skills:
Pyspark, Sql, Python
Skills:
Pyspark, python, Oops, Flask, Pandas, Numpy, django, data