Lead- Data Engineer

Technical skills:

• Should have 6+ years of hands-on experience in Data engineering

• Excellent command over ETL tools covering Snowflake, Azure Data Lake, Databricks and Azure Data Factory

• Good command over writing Python Scripts for custom build data ingestion

• Proficient in SQL scripts (Stored procedure, Views, Functions, Joins)

• Experience with Snowflake

• Experience with DBT (for ETL transformation logic)

• Experience with FIVETRAN (for data ingestion)

• Good understanding of data warehouse fundamentals and principles, database modelling knowledge and experience • Experience utilizing data pipelines and connectors across different source systems

• Experience with cloud data integration tools and services related to Fivetran, DBT or similar data ingestion and data transformation tools

• Experience working Git version control, preferably GitHub

• Professional IT related education or equivalent (university) education (e.g., B.E/BTech/MCA/MSC) in a relevant subject, preferably in Computer Science is a preferred

• Experience working as ETL/ELT developer or a Data engineer

Requirements for the role:

• Analzse and organize raw data, providing datasets and sources for business reporting requirements 

• Implement data quality rules to enhance data reliability • Understand business needs and objectives

• Conduct complex data analysis and data modelling, interpreting trends and patterns

• Collaborate with data scientists and Reporting engineers to help resolve the reporting and Analytics needs

• Provide expertise and leadership in the development of new products, services, and processes from a Data Engineering technical perspective.

• Work independently adhering to high technical standards, taking ownership of projects ensuring they conform to departmental standards before going to QA.

• Work effectively within an agile, deadline-driven production process to complete project development with rapid, iterative cycles.

• Liaise with Team Leaders, Product and Project Managers to ensure projects are taken from creation to completion - on time and on budget. 

• Provide accurate estimates for existing and new projects alike. 

• Be an integral part of the code review process, providing constructive feedback against code produced by other team members.

• Ensure good communication is maintained with other team members/departments throughout project life cycles. 

• Write clear, clean, and concise scripts, SQL and transformation logic 

• Fix bugs and transformation problems rapidly

• Use lateral thinking and investigative techniques to solve challenges across various technical streams. 

• Participate in the ongoing development, adoption and promotion of Data Engineering standards 

• Investigate and participate in discussions with team members about best practices and help identify optimal technical solutions

• Actively contribute to departmental learning and standards within the team environment through involvement in presentations/demonstrations and participate in the continual drive for collaboration and knowledge sharing amongst the team 

• Keep abreast of the latest trends and emerging technologies within the Data Engineering community and continually develop skill