Senior QA Engineer

Home > Other programming jobs

Senior QA Engineer in Others

QAD, Inc.
Full time
Email

Others
10/13/2026
Report

Pune, India

Responsibilities

Design and maintain end-to-end business scenarios covering agent routing, tool and function calls, orchestration, handoffs, and multi-turn conversations.
Evaluate agent responses for correctness, relevance, groundedness, consistency, tone, and completeness using human and automated methods.
Build and maintain evaluation datasets, golden sets, edge cases, adversarial prompts, and automated regression suites.
Identify cross-agent, orchestration, context, integration, tool-call, looping, and runaway-behavior issues.
Test for hallucinations, prompt injection, jailbreaks, data leakage, and unsafe or out-of-scope responses.
Detect and quantify regressions caused by model, prompt, retrieval, or knowledge changes.
Triage issues with Engineering and Product, determine root causes, re-test fixes, and validate the end-user experience.
Document test strategy, evaluation results, and quality trends for global stakeholders.

Requirements

Bachelor's degree in software engineering, computer science, or a related field.
3-5 years of QA experience, including at least 1-2 years testing LLM-based applications, AI agents, or agentic workflows.
Hands-on experience evaluating non-deterministic outputs by defining pass/fail criteria, rubrics, and acceptance thresholds.
Experience building or maintaining AI evaluation or regression suites, including LLM-as-judge methods, golden datasets, or scoring rubrics.
Understanding of tool and function calling, multi-agent orchestration, planning, memory and context, and RAG.
Experience with test automation tools such as Selenium, Playwright, or TestComplete, and with API testing.

Benefits

Health and well-being programs supporting work-life balance.
Collaborative culture with growth and advancement opportunities.
Virtual-first work arrangement with primary work performed from home.
Occasional travel to a physical office may be required for collaboration, design, strategy, and alignment.
Compensation packages are based on experience and desired skill set.

Salary: DOE

QAD, Inc.

QAD, Inc. delivers innovative enterprise software solutions that streamline manufacturing operations and empower businesses to achieve greater efficiency and agility. Our platform integrates planning, execution, and analytics to help companies adapt quickly to market demands while maintaining hig...

Similar positions

Sr Software Engineer - Builder Tools

Human Interest
Full time

Remote
11/12/2026
Salary: $190k-$230k

United States

Principal Engineer AI Native Engineering

Flexera
Full time

Remote
11/12/2026

India

Machine Learning Engineering Manager (Power)

Kpler
Full time

Remote
11/12/2026

Analytics Engineer II - Paid Search

StubHub
Full time

USA
11/12/2026
Salary: $140k - $180k/yr

Los Angeles, CA

Fresher Test Engineer QA QC

KMS Technology
Full time

Others
11/12/2026

Đà Nẵng, Vietnam