Role purpose
The Data Engineer is responsible for designing, building, and maintaining scalable data pipelines, integrations, and data warehouse structures that support organizational analytics and reporting. The role ensures high-quality, reliable, and secure data flow across systems, enabling business units to make data-driven decisions aligned with Kun Sports strategic objectives.
KEY ACCOUNTABILITIES & ACTIVITIES
- Data Pipeline Development & Management
- Design, build, and maintain scalable ETL/ELT pipelines for ingestion, cleaning, transformation, and loading of structured and unstructured data.
- Building workflows using Python frameworks such as FastAPI, Redis, RabbitMQ etc.
- Automate and orchestrate data workflows using Airflow, DBT, Prefect, or cloud-native schedulers.
- Implement automated job monitoring, error logging, and alerting mechanisms.
- Optimize pipeline performance to reduce latency, processing time, and resource consumption.
- Develop real-time streaming pipelines using Kafka, Kinesis, or similar technologies.
- Data Warehouse & Database Architecture
- Build and maintain enterprise data warehouse schemas (star, snowflake, normalized).
- Model data structures to support dashboards, analytics, and predictive models.
- Set up database indexing, partitioning, and clustering to enhance query performance.
- Maintain metadata repositories, data catalogs, and lineage tracking.
- Data Quality, Governance & Compliance
- Establish validation rules, anomaly detection, and data quality scoring frameworks.
- Ensure compliance with internal data governance, security policies, and data retention rules.
- Implement RBAC (role-based access control) and least-privilege access policies for all environments.
- Monitor and maintain data accuracy, completeness, timeliness, and integrity.
- Document data sources, transformation logic, and quality standards.
- System Integration & API Development
- Develop and maintain robust integrations with HRMS (Menaitech), CRM, ERP, POS, Sentinel, fitness systems, and external vendors.
- Build and maintain RESTful APIs and data endpoints for cross-department usage.
- Troubleshoot API failures, authentication issues, and versioning conflicts.
- Create scalable integration architectures for future system expansion.
- Analytics, Reporting & Business Support
- Enable BI team with optimized datasets for Power BI, Tableau, Looker, or internal dashboards.
- Support advanced analytics, machine learning pipelines, and predictive modeling.
- Translate business and department requirements into data architecture solutions.
- Build reusable data models for Finance, HR, Operations, Sales, and Marketing.
- Provide deep-dive analysis and root cause investigations for data inconsistencies.
- Cloud Environment & Infrastructure
- Manage cloud storage, compute instances, networking, and access permissions (AWS/Azure/GCP).
- Set up CI/CD pipelines for data engineering deployments.
- Manage Terraform / IaC for provisioning infrastructure.
- Ensure cost optimization for cloud-based workloads and data storage.
- Maintenance, Documentation & Continuous Improvement
- Maintain detailed technical documentation for data pipelines, schemas, integration flows, APIs, and architecture.
- Conduct regular system maintenance, patch updates, and optimization.
- Create runbooks for troubleshooting common issues and recovery procedures.
- Conduct capacity planning and scalability assessments.
- Recommend and implement new tools, technologies, and best practices.
REQUIREMENTS FOR ROLE
Experience & qualifications
- Bachelor's Degree in Computer Science, Information Systems, Data Engineering, or related field.
- 46 years of experience in data engineering, data warehousing, or backend data infrastructure.
Knowledge & skills
- Strong SQL proficiency and experience with relational and NoSQL databases (e.g., PostgreSQL, MySQL, MongoDB).
- Hands-on experience with ETL tools, data orchestration (Airflow, AWS Glue, DBT, etc.).
- Experience with cloud data environments (AWS, Azure, or GCP).
- Strong proficiency in Python.
- Solid understanding of APIs, data modeling, and data architecture principles.
- Experience with BI tools connectivity (Power BI, Tableau, Looker).
- Strong troubleshooting, debugging, and problem-solving skills.
- Strong documentation, communication, and stakeholder management skills.
