Mercy Munyao

Detail-oriented AI Data Annotator and AI Model Evaluation Specialist with 5+ years of experience supporting AI training and data annotation projects since 2021. Experienced in image, video, audio and text annotation, multimedia evaluation, computer vision labeling, content evaluation, search relevance assessment and AI model evaluation. Experienced in reviewing and annotating visual and multimedia data, following detailed project guidelines, identifying errors and edge cases, performing quality assurance and maintaining high levels of accuracy and consistency across large-scale AI datasets.

Strong background in image and video annotation, audio transcription, speaker tagging, computer vision labeling, dataset validation and AI-generated content evaluation. Comfortable working independently in remote environments, learning new annotation platforms and applying detailed guidelines consistently. Previous experience includes Sovrano AI, Atlas Capture, iMerit, OneForma, Toloka Programs, RWS Group and Scale AI.

AI Evaluator and Specialist at Sovrano AI (2026-08 – Present)

• Conducted search relevance evaluations including query-result matching, intent satisfaction, landing page quality analysis and ranking assessment.
• Evaluated online and social media content for misinformation indicators, policy compliance, toxicity and sentiment classification.
• Evaluated and ranked LLM responses for helpfulness, factuality, safety, instruction-following, reasoning quality and tone.
• Tested advanced model reasoning and identified logical inconsistencies, hallucinations, edge-case failures and bias risks.
• Applied nuanced evaluation rubrics to improve alignment and response quality across AI systems.

AI Data Annotator at Atlas Capture (2021-04 – 2021-08)

• Performed AI data annotation tasks involving visual and multimedia datasets.
• Reviewed and annotated images and other data according to detailed project-specific guidelines.
• Identified and labeled required visual elements accurately and consistently.
• Performed quality checks to identify incomplete, inaccurate or inconsistent annotations.
• Followed strict annotation instructions and adapted to changing project requirements.
• Maintained accuracy and consistency while completing high-volume annotation tasks.
• Worked independently in a remote environment while meeting project quality standards.

AI Data Annotator at iMerit (2025-09 – 2025-12)

• Participated in AI training and data annotation projects involving images, video and multimedia data.
• Annotated visual datasets according to detailed project guidelines and annotation standards.
• Reviewed image and video content and identified relevant objects, actions, attributes and other required data points.
• Performed quality-control checks and corrected annotation errors.
• Maintained consistency across large datasets while working under project quality requirements.
• Adapted to new annotation guidelines, workflows and project requirements.
• Contributed to the development of high-quality datasets for AI and machine-learning applications.

Gen AI Model Trainer and AI Evaluation Specialist at OneForma (2025-03 – Present)

• Evaluated and ranked LLM responses across helpfulness, factuality, safety, instruction-following, reasoning quality and tone dimensions.
• Conducted preference ranking and comparative response analysis for reinforcement learning from human feedback (RLHF) workflows.
• Provided detailed evaluator feedback used to guide future model training and optimization priorities.
• Specialized in evaluating complex prompts involving reasoning, multi-step instruction-following and safety-sensitive outputs.
• Contributed to reducing unsafe or low-quality responses through structured human feedback and quality review.
• Applied nuanced evaluation rubrics to improve alignment and response quality across AI systems.

Search Specialist and AI Evaluator at Toloka Programs (2024-03 – 2025-12)

• Conducted search relevance evaluations including query-result matching, intent satisfaction, landing page quality analysis and ranking assessment.
• Evaluated online and social media content for misinformation indicators, policy compliance, toxicity and sentiment classification.
• Maintained high agreement scores with benchmark labels through disciplined guideline adherence and calibration.
• Reduced review rejections by 20% after implementing a structured QA checklist and edge-case review process.
• Documented annotation decisions and reasoning patterns to improve consistency across evaluator teams.
• Supported A/B testing and ranking experiments for recommendation systems and search optimization projects.

AI Data Annotator and Content Evaluator at RWS Group (2022-04 – 2023-09)

• Participated in AI training and annotation projects involving text evaluation, response ranking and quality review.
• Evaluated AI-generated outputs using detailed project guidelines and performance benchmarks.
• Assisted with content moderation, policy compliance, safety classification and data validation tasks.
• Conducted editorial QA and verified annotation consistency across datasets.
• Supported multilingual and localization-focused AI evaluation workflows.

Data Annotator at Scale AI (2021-01 – 2021-09)

• Labeled and reviewed audio datasets including ASR transcription, speaker diarization tagging and intent classification.
• Achieved 98%+ accuracy on gold-standard evaluation tasks and calibration reviews.
• Performed computer vision annotation including segmentation, polygons, key points and bounding boxes on complex datasets.
• Exceeded weekly annotation targets by 10–20% while maintaining strict quality KPIs.
• Reduced downstream QA rejections by 25% through careful validation and review processes.
• Documented annotation edge cases and contributed to improved guideline clarity for future evaluators.

Bachelor of Education in English / Literature – Kenyatta University (2012-09 – 2016-12)