Data Engineer

Responsibilities

. Responsible for designing, developing, and maintaining data solutions for data generation, collection, and processing.

. Creating data pipelines, ensuring data quality, and implementing ETL processes to migrate and deploy data across systems.

. You are expected to be a subject matter expert, collaborate and manage the team to perform effectively.

. Engage with multiple teams, and contribute to key decisions while providing solutions to problems for your immediate team and across multiple teams.

. Query, clean, and transform datasets using SQL and Python on Databricks / Pyspark .

. Ensure data quality and consistency across systems in accordance with government Instruction Manual (IM8) standards.

. Build, enhance, and maintain dashboards for data monitoring, performance reporting, and user behaviour analytics.

. Partner with business stakeholders to understand data requirements and translate them into impactful visual insights.

. Documentation & Knowledge Management

. Develop clear and comprehensive user guides for dashboards, charts, and data workflows in Databricks

. Maintain technical documentation (e.g., on Confluence) for analytics processes, standards, and best practices.

Requirements

. Bachelor's degree in Computer Science, Engineering, or related field.

. At least 4-5 years of relevant working experience

. Proficiency in SQL and Python for data processing and automation.

. Hands-on experience in dashboard tools such as Tableau, Power BI, or equivalent visualisation tools.

. Familiarity with Databricks or similar big data/analytics platforms.

. Ability to produce high-quality technical documentation.

. Experience in system testing, SIT/UAT, or quality assurance processes.

. Knowledge of Spark for distributed data processing.

. Understanding of Git or other version control systems.

. Advanced proficiency in Data Engineering is required.

. Advanced proficiency in Data Access is recommended.

. Advanced proficiency in Data Migrations, Data Pipelines, and Data Processing is suggested.

Licence no: 12C6060