Senior Quality Assurance Engineer

QA Engineer – Agentic Gateway Testing

Job Summary

We are looking for a QA Engineer specializing in Agentic Gateway Testing to ensure the reliability, security, performance, and observability of enterprise AI agent workflows and tool integrations.

The role will focus on testing MCP-based interactions, multi-step agent workflows, tool discovery and invocation, authorization controls, performance characteristics, and end-to-end traceability across an enterprise agentic platform.

The ideal candidate will have strong experience in QA automation, AI/ML or LLM-based systems, API and integration testing, security testing, performance testing, and Python test automation, with hands-on exposure to agentic workflows and tool-calling architectures.

Project Overview

About the Client

Our client is a large global enterprise operating across multiple markets, with a complex technology landscape and a strong focus on digital transformation and innovation. The organization is actively investing in modern cloud, data, and AI capabilities to enable scalable, secure, and highly automated solutions across its business.

About the Project

The project is focused on building an enterprise-grade AI Agent Platform from the ground up. The platform will provide a standardized foundation for developing, deploying, orchestrating, securing, and observing AI agents across multiple teams and use cases.

The initiative covers the full agent lifecycle and brings together agent orchestration, observability, security, governance, integrations, evaluation, and platform engineering.

Engineers joining the project will have the opportunity to influence key architectural and technical decisions and contribute to a new platform rather than maintaining an existing solution.

Key Responsibilities

• Design and execute comprehensive tests for agentic workflows, including tool discovery, tool invocation, multi-step agent interactions, and end-to-end workflows.
• Validate MCP protocol behavior, including tool-list aggregation, tool-call routing, request/response handling, and streaming responses.
• Test agent authorization and authentication controls, including unauthorized tool-access attempts and policy-enforcement scenarios.
• Validate that agents can access only the tools and resources permitted by their configured policies.
• Design and execute performance and load tests for agentic workloads, including tool-discovery latency, concurrent agent sessions, and agent-to-agent communication latency.
• Build and maintain automated tests using Python and pytest, including asynchronous test scenarios, mock MCP servers, and agent workflow simulations.
• Develop test strategies for complex multi-step and multi-agent workflows.
• Validate agentic observability, including trace completeness, agent-step logging, correlation IDs, and audit trails.
• Perform end-to-end testing across agent, gateway, MCP server, and downstream tool integrations.
• Validate error handling, retries, timeouts, failures, and recovery mechanisms across agentic workflows.
• Identify and document reliability, security, performance, and integration defects and work closely with engineering teams to resolve them.
• Develop regression suites to ensure new gateway, agent, and tool integrations do not introduce existing defects.
• Support testing across multiple environments and participate in release validation.
• Validate audit and compliance requirements for agent interactions and tool access.
• Contribute to QA standards, test automation frameworks, and quality engineering practices for the enterprise agent platform.

Required Skills & Experience

• 4+ years of experience in QA Engineering, Test Automation, or Quality Engineering, preferably supporting AI/ML or complex distributed systems.
• Hands-on experience testing LLM or AI agent workflows, including tool calling and multi-step agent interactions.
• Strong experience with API, integration, and end-to-end testing.
• Experience testing authentication and authorization/security controls.
• Experience performing performance testing and benchmarking for cloud-based or distributed services.
• Strong Python programming skills for automated testing.
• Hands-on experience with pytest, including asynchronous testing.
• Experience creating mocks, stubs, simulators, or test doubles for external services.
• Strong understanding of distributed systems, APIs, HTTP/REST, and asynchronous communication.
• Experience designing and executing functional, regression, integration, negative, security, and performance test scenarios.
• Strong analytical and troubleshooting skills.

Nice to Have

• Hands-on experience testing the MCP (Model Context Protocol).
• Experience with MCP tool discovery, tool routing, streaming responses, and server/client interactions.
• Experience testing agent frameworks such as LangChain, LangGraph, or CrewAI.
• Experience testing AWS Bedrock Agents or other managed AI agent platforms.
• Experience with chaos engineering and resilience testing for distributed or agentic systems.
• Experience validating CloudTrail audit logs.
• Experience with OpenTelemetry traces and distributed tracing.
• Experience with agent evaluation and LLM testing frameworks.
• Experience testing AI-specific risks such as prompt injection, tool misuse, data leakage, and unsafe tool access.

Technical Environment

• Programming: Python
• Test Automation: pytest, pytest-asyncio
• AI/Agents: LLMs, AI Agents, Multi-step Workflows
• Protocol: MCP (Model Context Protocol)
• Agent Frameworks: LangChain, LangGraph, CrewAI
• Cloud: AWS
• AI Services: AWS Bedrock / Bedrock Agents
• Observability: OpenTelemetry, distributed tracing, audit logs
• Security: Authentication, authorization, policy enforcement
• Testing: Functional, Integration, E2E, Regression, Performance, Security, Resilience
• Architecture: Agent → Gateway → MCP Server → Tools/Services

Key Testing Areas

The successful candidate will be expected to develop and execute test coverage across:

Agentic Workflow Testing

• Tool discovery
• Tool selection and invocation
• Multi-step agent workflows
• Agent-to-agent interactions
• Error handling and recovery
• Timeout and retry behavior

MCP Testing

• Tool-list aggregation
• Tool discovery
• Tool-call routing
• Request/response validation
• Streaming responses
• MCP server failure scenarios
• Invalid and malformed requests

Security Testing

• Authentication validation
• Authorization enforcement
• Unauthorized tool-access attempts
• Policy enforcement
• Privilege escalation scenarios
• Sensitive-data exposure
• Tool-access boundary testing

Performance Testing

• Tool-discovery latency
• Tool-call latency
• Concurrent agent sessions
• Gateway throughput
• Agent-to-agent latency
• Response streaming performance
• Load and stress testing

Observability Testing

• End-to-end trace completeness
• Agent-step logging
• Tool invocation logs
• Correlation and trace IDs
• Audit trail accuracy
• Error and failure visibility

Education

• Bachelor's degree in computer science, Software Engineering, Information Technology, Engineering, or a related field.

Why This Position

• Work on a greenfield enterprise AI Agent Platform.
• Help establish quality and testing standards for emerging agentic technologies.
• Work at the intersection of QA, AI, cloud, security, distributed systems, and automation.
• Build automated testing capabilities for complex multi-step AI workflows.
• Gain hands-on exposure to MCP, LLM agents, agent gateways, and enterprise AI infrastructure.
• Collaborate with AI, platform, security, DevOps, and software engineering teams.
• Contribute to a strategic AI transformation initiative within a large global enterprise.