The Data Engineer will be responsible for designing, building, and maintaining secure, scalable data pipelines and analytics platforms. The role combines data engineering, analytics enablement, and AI integration, leveraging relational databases, Google BigQuery, modern BI tools, and Python-based machine learning libraries. The ideal candidate will enable data-driven decision-making by ensuring high-quality data ingestion, transformation, analytics, and visualization across the organization.
1. Data Engineering & Ingestion
- Design, build, and maintain secure data ingestion pipelines from multiple internal and external data sources.
- Implement ETL/ELT processes to move and transform data into Google BigQuery.
- Ensure data quality, validation, lineage, and governance across pipelines.
- Optimize data pipelines for performance, scalability, and cost efficiency.
2. Databases & Data Warehousing
- Design and manage relational databases using PostgreSQL or MySQL.
- Model data for analytics and reporting (star/snowflake schemas).
- Develop and optimize BigQuery datasets, tables, partitions, and views.
- Manage data access controls and security policies.
3. Data Analytics & Reporting
- Enable and support analytics and reporting using Power BI Pro (with Copilot) and Looker Enterprise.
- Build semantic models, datasets, and data sources for BI tools.
- Collaborate with stakeholders to define KPIs, metrics, and dashboards.
- Ensure performance, reliability, and accuracy of reports and dashboards.
4. Data Visualization
- Develop custom data visualizations using Chart.js and D3.js.
- Integrate visualizations into web applications and internal portals.
- Ensure visual clarity, usability, and data accuracy.
5. AI & Machine Learning Integration
- Develop data processing and feature engineering pipelines using Python.
- Build and evaluate machine learning models using Scikit-learn and XGBoost.
- Perform data analysis and manipulation using Pandas.
- Integrate external AI services such as OpenAI APIs into data workflows and applications.
- Support experimentation, model deployment, and performance monitoring.
6. Security, Quality & Operations
- Implement secure data access, encryption, and compliance best practices.
- Monitor data pipelines, troubleshoot failures, and optimize reliability.
- Document data architectures, pipelines, and operational procedures.
- Collaborate with DevOps and security teams to ensure stable data platforms.
Technical Skills
- Minimum 3 years of experience as a Data Engineer.
- Strong experience with:
- PostgreSQL or MySQL
- Google BigQuery
- Secure data ingestion from multiple data sources
- Python for data engineering and analytics
- Pandas, Scikit-learn, XGBoost
- Power BI Pro (Copilot) and Looker Enterprise
- Hands-on experience with:
- Data modeling and warehousing concepts
- SQL performance tuning and optimization
- BI semantic layers and dashboards
- Data visualization libraries (Chart.js, D3.js)
- API-based data ingestion and integration (including OpenAI API)
Soft Skills
- Strong analytical and problem-solving skills.
- Ability to translate business requirements into data solutions.
- Good communication and documentation abilities.
- Ability to work in cross-functional, agile teams.
