Overview

The SOC Lead Engineer, DevOps is a specialized technical role responsible for managing and optimizing large scale platform and data operations within the Security Operations Center (SOC). This individual is proficient in a wide array of big data and DevOps/Platform related tools and technologies. This position requires at least 8 years of experience in DevOps, Big data environments and a solid understanding of Operating Systems, APIs, (ETL) processes, and scripting.

Responsibilities

Key Responsibilities

Manage and optimize large-scale data operations and big data technologies.
Develop scripts and leverage automation tools to streamline operations and minimize manual intervention. This includes automating deployment processes, system configurations, and routine maintenance tasks.
Involve in architectural discussions and provide the best solution following standard design patterns.
Provide expert support for system management, monitoring, and troubleshooting. Identify, diagnose, and resolve system issues to minimize downtime.
Develop and maintain ETL/ELT processes, standards, and procedures for data management to ensure data accuracy and integrity.
Continuously monitor, manage, optimize, and report on the performance of SOC solutions, identifying and resolving any issues or bottlenecks, incidents, and resolutions.
Develop, implement, and maintain strategies for data collection, analysis, data processing, and data security using tools like Flink, Spark, CRIBL, Hadoop, ELK, Cloudera, and Data Lake.
Ensure the optimized operations of big data tools within the SOC.
Collaborate with different teams to understand data requirements and ensure data availability and quality.
Participate in continuous process improvements to increase SOC efficiency and effectiveness.
Contribute to SOC strategy and initiatives related to big data, platform and devops engineering and management.

Qualifications

Job Specifications

Skills/Certifications (Technical & Non-Technical)

Skills

Solid understanding of hosted and cloud platforms (AWS, Azure, Google Cloud) and their services.
Proficiency in working with various Operating Systems (OS) and APIs.
Proficiency in big data technologies and distributed compute systems such as ELK, CRIBL, Apache Kafka, Data Lake House, and Cloudera.
Experience with microservices architecture and containerization technologies (e.g., Docker, Kubernetes).
Extensive knowledge in Git and automated build and deployment tools (such as Jenkins, Azure DevOps Pipelines) and utilizing Ansible, Terraform, etc. in hybrid environments.
Knowledge of Gitops process and experience with tools like Argo CD, Flux CD etc.
Strong programming skills, particularly in Python and Bash scripting, for automating data processes and troubleshooting/debugging.
Extensive knowledge of data management and security principles, including data processing, normalization, data quality, data encryption and database management.
Good to have an understanding or hands-on experience with ETL/ELT processes, Big Data tools and Data Lake.
Proficiency in SQL and NoSQL for data manipulation and retrieval.
Excellent problem-solving skills, with the ability to make decisions under pressure.
Stakeholder management and experience in building high performing teams.
High proficiency in written and verbal communication.
Exceptional collaboration and team development skills including capacity planning.

Certifications

Cloud-related certifications like AWS Certified Solutions Architect, Google Professional Cloud Architect, or Microsoft Certified: Azure Solutions Architect Expert.
Certified Kubernetes Administrator (CKA) or Certified Kubernetes Application Developer (CKAD) would be beneficial.
Cisco CCNA R&S
CompTIA Security+, LPIC certifications or similar would be beneficial.

Minimum Work Experience

At least 8 years of experience in big data and platform engineering in a cybersecurity context.
Considerable experience working with a range of big data technologies and tools, cloud environments, and practical knowledge of applying best practices of data security

Education

Bachelor's degree in computer science, Information Technology, Cybersecurity, or related field.
Master's degree in data science or related is a bonus.

Lead SOC Engineer (Devops)