Prajna Parth

Quality Evaluation specialist with 4+ years assessing and validating AI model outputs, generative AI responses, and data annotation quality against defined rubrics and standards, backed by a strong software QA foundation. At SIGMA AI, led quality evaluation of data annotation and RLHF/generative AI outputs against accuracy, consistency, and safety criteria, directing cross-functional teams of 50–80 evaluators across multiple locales. Experienced in evaluating text-to-speech and voice-over output for naturalness, pacing, tone, accent, and pronunciation.

Skilled in building structured evaluation frameworks, calibrating rater/evaluator judgments, and turning evaluation findings into actionable feedback that improves model performance, alongside traditional defect tracking (Bugzilla, Jira, Redmine, Watson). Comfortable working independently on functional, exploratory, and usability test cycles, comparing observed results against expected outcomes and providing structured feedback via reports, surveys, and interviews.

Quality Evaluator / Project Manager — Generative AI Data Quality - SIGMA AI - London, Hammersmith

(2022-10)

Evaluated data annotation and generative AI outputs against quality rubrics covering accuracy, consistency, and safety, directly shaping model training data integrity.

• Designed and applied structured evaluation criteria across multiple concurrent generative AI projects, directing teams of 50–80 evaluators and annotators across diverse locales.
• Calibrated evaluator judgments to improve inter-rater consistency and reliability of quality scores.
• Translated subjective qualities (e.g. naturalness, tone, fluency) into scored rubrics with clear definitions and examples for each score level, refining guidelines based on edge cases and evaluator disagreements.
• Evaluated text-to-speech and voice-over output for naturalness, pacing, intonation, accent, and pronunciation accuracy, flagging issues such as robotic prosody, mispronunciations, and unnatural pauses.
• Rated audio and video outputs and ran side-by-side comparisons of model responses, documenting the reasoning behind each preference to guide model improvement.
• Converted evaluation findings into structured quality feedback that supported iterative development and refinement of AI models.
• Owned delivery timelines and evaluation standards across AI workflows, ensuring outputs met client and model-training benchmarks.
• Partnered with data annotators, engineers, and stakeholders to resolve quality issues and continuously optimize evaluation outcomes.

Localization Tester - PTW - London, United Kingdom

(2021-12)

• Tested games across multiple locales and platforms (Android & iOS), identifying linguistic and implementation defects.
• Logged and tracked defects using Jira and Redmine, coordinating fixes with development teams.

Manual Software Tester - QA InfoTech Pvt. Ltd. - Noida, UP, India

(2009-01 - 2010-12)

Client: Adobe (Community Help, Photoshop Elements 8 & 9)

• Executed functional and regression test cases across multiple locales for Adobe products.
• Verified GUI consistency and multilingual support; tracked and prioritized bugs using Watson.
• Ensured critical functionalities were thoroughly tested and documented ahead of release.

MSc - Bioinformatics - Banasthali Vidyapith (2007)

MSc - Kumaun University (2004)

BSc - Kumaun University (2002)