Overview
The SOC Lead Engineer, DevOps is a specialized technical role responsible for managing and optimizing large scale platform and data operations within the Security Operations Center (SOC). This individual is proficient in a wide array of big data and DevOps/Platform related tools and technologies. This position requires at least 8 years of experience in DevOps, Big data environments and a solid understanding of Operating Systems, APIs, (ETL) processes, and scripting.
Responsibilities
Key Responsibilities
- Manage and optimize large-scale data operations and big data technologies.
- Develop scripts and leverage automation tools to streamline operations and minimize manual intervention. This includes automating deployment processes, system configurations, and routine maintenance tasks.
- Involve in architectural discussions and provide the best solution following standard design patterns.
- Provide expert support for system management, monitoring, and troubleshooting. Identify, diagnose, and resolve system issues to minimize downtime.
- Develop and maintain ETL/ELT processes, standards, and procedures for data management to ensure data accuracy and integrity.
- Continuously monitor, manage, optimize, and report on the performance of SOC solutions, identifying and resolving any issues or bottlenecks, incidents, and resolutions.
- Develop, implement, and maintain strategies for data collection, analysis, data processing, and data security using tools like Flink, Spark, CRIBL, Hadoop, ELK, Cloudera, and Data Lake.
- Ensure the optimized operations of big data tools within the SOC.
- Collaborate with different teams to understand data requirements and ensure data availability and quality.
- Participate in continuous process improvements to increase SOC efficiency and effectiveness.
- Contribute to SOC strategy and initiatives related to big data, platform and devops engineering and management.
Qualifications
Job Specifications
Skills/Certifications (Technical & Non-Technical)
Skills
- Solid understanding of hosted and cloud platforms (AWS, Azure, Google Cloud) and their services.
- Proficiency in working with various Operating Systems (OS) and APIs.
- Proficiency in big data technologies and distributed compute systems such as ELK, CRIBL, Apache Kafka, Data Lake House, and Cloudera.
- Experience with microservices architecture and containerization technologies (e.g., Docker, Kubernetes).
- Extensive knowledge in Git and automated build and deployment tools (such as Jenkins, Azure DevOps Pipelines) and utilizing Ansible, Terraform, etc. in hybrid environments.
- Knowledge of Gitops process and experience with tools like Argo CD, Flux CD etc.
- Strong programming skills, particularly in Python and Bash scripting, for automating data processes and troubleshooting/debugging.
- Extensive knowledge of data management and security principles, including data processing, normalization, data quality, data encryption and database management.
- Good to have an understanding or hands-on experience with ETL/ELT processes, Big Data tools and Data Lake.
- Proficiency in SQL and NoSQL for data manipulation and retrieval.
- Excellent problem-solving skills, with the ability to make decisions under pressure.
- Stakeholder management and experience in building high performing teams.
- High proficiency in written and verbal communication.
- Exceptional collaboration and team development skills including capacity planning.
Certifications
- Cloud-related certifications like AWS Certified Solutions Architect, Google Professional Cloud Architect, or Microsoft Certified: Azure Solutions Architect Expert.
- Certified Kubernetes Administrator (CKA) or Certified Kubernetes Application Developer (CKAD) would be beneficial.
- Cisco CCNA R&S
- CompTIA Security+, LPIC certifications or similar would be beneficial.
Minimum Work Experience
- At least 8 years of experience in big data and platform engineering in a cybersecurity context.
- Considerable experience working with a range of big data technologies and tools, cloud environments, and practical knowledge of applying best practices of data security
Education
- Bachelor's degree in computer science, Information Technology, Cybersecurity, or related field.
- Master's degree in data science or related is a bonus.
