Senior Data Engineer
Job Description
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and design appropriate solutions
- Work closely with IT and infrastructure teams to deploy and maintain data storage and processing systems, including databases, data warehouses, and cloud infrastructure
- Design, build, and maintain robust, scalable data pipelines to ingest, process, and transform data from various sources, including internal databases and external APIs
- Optimize data pipelines for performance, scalability, and reliability, ensuring timely and accurate delivery of data
- Implement data validation and quality checks to identify and rectify any issues in data pipelines, ensuring data integrity
- Explore and evaluate new technologies, tools, and frameworks to improve data engineering processes and capabilities
- Perform data modeling and schema design to support analytical and reporting requirements
- Design and build data marts and business layers to facilitate efficient data access and analysis
- Bachelor's degree in Computer Science, Information Systems, or other related technical field or equivalent work
- Proven experience as a Data engineer or similar role, with at least 4 years of experience
- Advanced SQL knowledge/experience - complex queries and query optimization
- Proficiency in programming languages such as Python
- Experienced in the development of data warehouses and proficient in data modeling
- Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform
- Experience in building data streams using technologies such as Kafka, Flink, or Spark Streaming is considered a plus
- Strong problem-solving skills and attention to detail
- The ability and willingness to learn new technologies on the job
- Self driven, highly motivated, fast learner, ambitious and creative
More Info
About Company
Skills:
Power Bi, Data Warehousing, Tableau, Qlikview, Data Modeling, Azure, Python, Sql, AWS, ETL processes
Skills:
snowflake , BigQuery, Scala, Kafka, Sql, Gcp, Terraform, Docker, Spark, Databricks, Kubernetes, Python, AWS, Airflow, Flink, Dagster
Skills:
Power Bi, Data Warehousing, Qlikview, Tableau, Data Modeling, Azure, Python, Sql, AWS, ETL processes
Skills:
Power Bi, Data Warehousing, Qlikview, Tableau, Data Modeling, Azure, Python, Sql, AWS, ETL processes
Skills:
data engineering , Metadata Management, Azure Data Factory, Data Integration, Pyspark, Data Profiling, Etl Development, Azure Synapse Analytics, Python, Sql, Data quality frameworks