Senior Data Engineer

Job description

Description: The Senior Data Engineer is responsible for leading the design, development, and optimization of data infrastructure and systems to meet complex business needs. They play a strategic role in developing and managing large-scale data pipelines, ensuring scalability, reliability, and performance. As a key technical expert, the Senior Data Engineer collaborates with cross-functional teams to deliver innovative data solutions and ensures adherence to best practices in data management and governance.

Responsibilities

• Leads the design and development of advanced data pipelines, ensuring seamless ETL/ELT of data into enterprise storage systems.
• Oversees data integration from diverse sources, maintaining data consistency, integrity, and compliance with enterprise standards.
• Transforms raw data into usable formats through cleansing, aggregation, filtering, and enrichment techniques.
• Optimizes data pipelines and workflows for scalability, performance, and cost efficiency.
• Establishes robust data validation, quality checks, and error-handling mechanisms to ensure enterprise data integrity.
• Mentors junior data engineers, promoting technical growth and team development.
• Implements and enforces best practices in data governance, including metadata management and lineage tracking.
• Drives the adoption of real-time data streaming and processing workflows to enable advanced analytics and decision-making.
• Partners with stakeholders to identify and address complex data challenges, delivering scalable and innovative solutions.
• Stays updated with emerging data technologies, evaluating their potential to enhance enterprise capabilities.
• Develops and enforces data security protocols to protect sensitive information.

Job Requirements

Education

• A bachelor s degree in computer science, data science, software engineering, information systems, or related quantitative field; master s degree preferred

Experience

• 8+ years of data engineering experience, including expertise in data integration, pipeline optimization, and enterprise-scale data solutions.
• Proven leadership in implementing big data solutions (e.g., Snowflake, Databricks) and distributed data systems (e.g., Apache Spark, Flink).

Skills

• Advanced proficiency in Apache technologies (Kafka, Airflow, Spark) and programming languages (Python, Java, Scala).
• Expertise in data query tools (SQL, Hive) and database technologies (NoSQL, Hadoop, Teradata).
• Strong knowledge of cloud platforms (AWS, Azure, GCP) and modern data architectures.
• Experience with real-time streaming data workflows and AI/ML analytics initiatives.
• Exceptional analytical and problem-solving skills with debugging expertise in complex systems.
• Excellent communication skills to effectively collaborate with technical and non-technical stakeholders.
• Strong leadership abilities to mentor teams and influence cross-functional initiatives.