Pyspark Data Engineer

We have an Immediate Job opportunity for Pyspark Data Engineer position.

Job Role: Pyspark Data Engineer

Job Location: Dubai, UAE

Experience : 6 to 14 Years

Notice Period: Immediate to 30 days

About Company:

At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions. Synechron's progressive technologies and optimization strategies span end-to-end Artificial Intelligence, Consulting, Digital, Cloud & DevOps, Data, and Software Engineering, servicing an array of noteworthy financial services and technology firms. Through research and development initiatives in our FinLabs we develop solutions for modernization, from Artificial Intelligence and Blockchain to Data Science models, Digital Underwriting, mobile-first applications and more. Over the last 20+ years, our company has been honored with multiple employer awards, recognizing our commitment to our talented teams. With top clients to boast about, Synechron has a global workforce of 15,000+, and has 58 offices in 21 countries within key global markets. For more information on the company, please visit our website or LinkedIn communit

Diversity, Equity, and Inclusion:

Diversity & Inclusion are fundamental to our culture, and Synechron is proud to be an equal opportunity workplace and an affirmative-action employer. Our Diversity, Equity, and Inclusion (DEI) initiative Same Difference is committed to fostering an inclusive culture promoting equality, diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger, successful businesses as a global company. We encourage applicants from across diverse backgrounds, race, ethnicities, religion, age, marital status, gender, sexual orientations, or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements, mentoring, internal mobility, learning and development programs, and more. All employment decisions at Synechron are based on business needs, job requirements, and individual qualifications, without regard to the applicant's gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected.

Job Description:

5+ years of previous commercial experience as a leader in a data-driven role

5+ years of hands-on experience building data pipelines in production and ability to work across structured, semi-structured and unstructured data

3+ years of experience in ML pipeline for streaming/batch workflow

Ability to write clean, maintainable, and robust code in Python

Understanding and expertise of software engineering concepts and best practices

Knowledge of testing frameworks and libraries

Experience with analytics (descriptive, predictive, EDA), feature engineer, algorithms, anomaly detection, data quality assessment and python visualization libraries - e.g. matplotlib, seaborn or other

Comfortable with notebook and source code development - Jupyter, Pycharm/VScode

Hands-on experience of technologies like Python, Spark/Pyspark, Hadoop/MapReduce/HIVE, Pandas etc.

Familiarity with query languages and database technologies, CI/CD, testing and validation of data and software

Tech stack and activities that you would use and preform on a daily basis

Required Skills:

Python

Spark (PySpark)

Jupyter

SQL and No-SQL DBMS

Git (as source code versioning and CI/CD)

Exploratory Data Analysis (EDA)

Imputation Techniques

Data Linking / Cleansing

Feature Engineering

Apache Airflow/ Jenkins scheduling and automation, Github and Github Actions

Qualifications:

Bachelor's or Master's degree in computer science, Information Systems, or related field

Soft Skills:

Excellent communication and leadership skills.

Strong interpersonal and collaboration skills.

Ability to work under pressure and meet tight deadlines.

Positive attitude and strong work ethic.