Data Engineer - Python

Required experience, skills

6+ years of hands-on data engineering development experience in Python Advanced knowledge of Python and Linux shell scripting Proficiency in SQL and Data Structures, experience in handling (transformations, read / write) huge volume of data using Python

Experience in performance improvement on data pipelines and a good understanding of Volume, Variety, Veracity, Velocity, and Value concepts in Big Data

• Good knowledge of algorithms and software design principles
• Experience in CI/CD pipelines with one of: Azure DevOps, AWS Code Pipeline, GitHub Actions Familiarity with Docker, Kubernetes, and cloud services (Azure, AWS, GCP) Proficient with Kafka, Flink, RabbitMQ or similar technologies Self-motivated, able to self-improve, learn new and complex Technologies
• In-depth knowledge and experience in agile processes and principles
• Summary: This JD is targeting a senior-level data engineer with extensive experience in building scalable data infrastructures, working with large datasets, optimizing performance, and applying cutting-edge technologies
• Strong Python and cloud experience are critical, as well as proficiency in Big Data tools and design patterns
• You will also need to be proactive in improving systems and aligning your work with business goals