Experience Level: Mid-Senior (5+ years)
Responsibilities
- Design, develop, and maintain scalable data pipelines on AWS.
- Implement and optimize Medallion Architecture for data processing and analytics.
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
- Ensure data quality, integrity, and security across all data pipelines.
- Monitor and troubleshoot data pipeline performance and reliability.
- Develop and maintain documentation for data engineering processes and solutions.
- Stay updated with the latest trends and best practices in data engineering and AWS technologies.
Requirements
- Bachelor's degree in Computer Science, Information Technology, or a related field.
- 5+ years of experience in data engineering, with a focus on AWS.
- Proven experience with Medallion Architecture implementation.
- Strong proficiency in AWS services such as S3, Redshift, Glue, Airflow.
- Experience with data modelling, ETL processes, Delta Tables, and data warehousing.
- Proficiency in programming languages such as Python or Spark.
- Strong SQL skills and experience with relational databases.
- Excellent problem-solving skills and attention to detail.
- Strong communication and collaboration skills.
