AI Quality Engineer / QE Consultant Playwright & GenAI Testing

About the Company

Greetings from Teamware Solutions a division of Quantum Leap Consulting Pvt. Ltd

Teamware Solutions (a division of Quantum Leap Consulting Pvt. Ltd.) is a 20+ year industry leader providing IT staffing and consulting services, supporting 4000+ professionals across India, the USA, the Middle East, and APAC for leading GCC clients.

About the Role

We are hiringan AI Quality Engineer / QE Consultant – Playwright & GenAI Testing

• Location: Bangalore
• Work Model: Hybrid
• Experience: 5 to 8 years
• Notice Period: Immediate - 15 days

Role Combination looking for

QE Consultant- Playwright + API Automation + AI/GenAI Testing + LLM Evaluation + CI/CD

Responsibilities

We are seeking a highly motivated Quality Engineering (QE) Consultant with 4–6 years of experience in software testing, automation, and emerging AI technologies. The ideal candidate will have hands-on experience testing AI/GenAI applications, APIs, and web applications, along with strong expertise in Playwright automation and AI evaluation frameworks.

This role will focus on ensuring the quality, reliability, safety, and performance of AI-powered solutions by designing comprehensive test strategies, implementing automation frameworks, validating AI model behavior, and establishing evaluation mechanisms for large language model (LLM)-based applications.

The candidate will collaborate closely with Product Owners, Developers, Data Scientists, and AI Engineers to drive quality throughout the software development lifecycle.

• AI/GenAI Testing: Test LLM applications, AI agents, copilots, and conversational systems for accuracy, relevance, groundedness, hallucinations, consistency, and safety.
• Playwright Automation: Build and maintain Playwright frameworks for UI, API, and E2E automation.
• API Testing: Perform API testing using Postman, REST Assured, and Playwright, including authentication, validation, and integration testing.
• AI Evaluation: Define AI quality metrics such as relevancy, semantic similarity, groundedness, precision/recall, and human evaluation.
• CI/CD: Integrate automation into CI/CD pipelines and support shift-left QE practices.
• Collaboration: Work with Product, Engineering, Data Science, and AI teams to identify risks and improve quality.

Experience

• 4–6 years of experience in Quality Engineering, Software Testing, or Test Automation.
• Hands-on experience testing AI/Generative AI applications.
• Experience with modern web application testing and API testing.

Technical Skills

• Strong expertise in Playwright automation.
• Hands-on experience with API testing and automation.
• Experience validating AI/LLM-based applications and AI agents.
• Good understanding of AI evaluation methodologies and testing techniques.
• Familiarity with:
• M365 Copilot
• OpenAI/Azure OpenAI
• Claude
• Gemini
• Agentic AI frameworks
• Knowledge of test management tools such as Azure DevOps (ADO), Jira, or similar platforms.
• Experience working with CI/CD pipelines and DevOps practices.

Preferred Skills

• Exposure to Python or TypeScript for automation and AI testing.
• Experience with AI evaluation frameworks such as:
• RAG evaluation
• Prompt evaluation
• LLM benchmarking
• Human feedback-based evaluation
• Knowledge of Responsible AI principles, AI governance, and model risk management.
• Experience testing Retrieval-Augmented Generation (RAG) systems and AI agents.

Education / Professional qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline..

Other Information

Number of interview rounds2

Please let me know if you are interested in this position and send me your resume to [Confidential Information]