We are seeking a highly motivated and detail-oriented Playwright Testing Engineer with experience in Generative AI (GenAI) evaluation testing. The ideal candidate will have a strong background in software quality engineering, test automation, and AI-driven testing methodologies.
This role involves validating AI-generated outputs, building automated test frameworks using Playwright, and ensuring the quality, safety, fairness, and reliability of AI-powered applications.
The successful candidate will work closely with Product, Engineering, Data Science, and AI teams to establish robust testing processes for GenAI features and contribute to responsible AI adoption across the organization.
Key Responsibilities :
- Design, develop, and maintain automated test frameworks using Playwright (JavaScript/TypeScript).
- Create and execute automated test cases for web applications and AI-powered features.
- Develop reusable evaluation test suites for Generative AI capabilities.
Validate AI-generated content for :
- Accuracy
- Relevance
- Clarity
- Consistency
- Inclusivity
- Bias Reduction
- Implement testing strategies for AI model outputs and user-facing GenAI experiences.
- Build and maintain AI evaluation pipelines using quantitative and qualitative metrics.
- Perform hallucination detection and response quality validation.
- Assess AI-generated content for privacy, security, compliance, and ethical considerations.
- Leverage AI productivity tools such as Cursor, AI Copilots, and Playwright MCP Servers to improve testing efficiency and coverage.
- Collaborate with developers, AI engineers, and product teams to define quality standards for AI features.
- Identify, document, track, and validate defects across application and AI layers.
- Participate in Agile ceremonies and contribute to continuous quality improvement initiatives.
- Advocate for responsible AI principles, transparency, and trustworthiness in AI-powered products.
SysMind