Brian Pathenal

LLM Evaluator and AI Evaluation Specialist with 5+ years of experience supporting the development, evaluation, and optimization of large language models through RLHF workflows, prompt engineering, response ranking, and structured quality assessment. Experienced in applying detailed evaluation rubrics to assess factuality, reasoning, instruction adherence, safety, and response quality across diverse domains. Skilled in designing prompts to test model capabilities, identifying failure modes, conducting adversarial testing, and providing clear, structured written feedback to improve model performance.

Strong background in AI model training, data annotation, quality assurance, and technical content development, with exceptional attention to detail and written communication skills. Highly proficient in Microsoft Office Suite (including PowerPoint) and Google Workspace (including Google Slides).

Advanced AI Trainer & LLM Evaluator - Mindrift - Remote

(2022-03 - 2025-11)

• Designed and evaluated 1,000+ complex computer science prompts, reasoning tasks, and question-answer pairs to train and rigorously assess large language model performance.
• Created high-quality training datasets aligned with AI model learning objectives across technical and analytical domains, ensuring factual accuracy and domain quality.
• Evaluated AI-generated responses using structured rubrics focused on accuracy, reasoning quality, instruction adherence, consistency, and completeness — identifying factual, logical, and presentation-level errors.
• Crafted targeted prompts to elicit specific model behaviors and assess performance under challenging and edge-case scenarios.
• Conducted adversarial testing to systematically identify weaknesses, limitations, and failure patterns in model outputs, producing clear written reports of findings.
• Reviewed and refined AI-generated content to improve technical accuracy, clarity, and educational value across documents and structured outputs.
• Supported RLHF data collection through annotation, response ranking, comparative evaluation, and preference-based feedback at scale.
• Collaborated with project teams to improve evaluation methodologies and dataset quality, translating evaluation findings into actionable model improvement guidelines.

Chatbot Trainer & AI QA Analyst - TELUS International - Remote

(2020-05 - 2022-01)

• Developed and curated conversational datasets to support chatbot training and language model optimization across multilingual evaluation projects.
• Ranked and compared AI-generated responses against domain-specific quality rubrics to improve response accuracy and model performance.
• Evaluated chatbot outputs for contextual relevance, coherence, safety, and user intent alignment, providing structured written feedback on identified errors.
• Analyzed recurring model errors and failure modes, producing detailed written assessments to support targeted model improvements.
• Collected, validated, and organized multilingual datasets for AI training and evaluation projects, maintaining high annotation quality standards.
• Identified edge cases and linguistic inconsistencies that impacted chatbot performance, documenting findings with clear, actionable recommendations.
• Collaborated with distributed teams to improve annotation standards and dataset reliability across evaluation workflows.

Master of Science - Computer Science - Strathmore University

Bachelor of Science - Computer Science - University of Nairobi