Katalina Tapia
AI Evaluation Specialist & Data Annotation Expert
Talent
PhiladelphiaMember since September 10, 2026
Prompt engineering and scenario designAI output evaluation, pairwise and multi-response rankingHelpfulness labelingRLHF and SFT data pipeline supportAdvanced English writingPythonText parsing and data cleaningJSON/CSV data handlingGoogle SheetsGoogle DocsNotionSlackGoogle CalendarGoogle DriveMicrosoft Office
·
Hire this person
Send a job offer directly to this candidate
Prompt engineering and scenario designAI output evaluation, pairwise and multi-response rankingHelpfulness labelingRLHF and SFT data pipeline supportAdvanced English writing
About
AI content evaluator and prompt engineer with two years of experience building evaluation prompts, ranking model outputs, and annotating data for RLHF pipelines.
Background in Information Technology gives a strong basis for judging tone, clarity, and instruction-following in model responses. Comfortable working independently across shifting rubrics and large data volumes.
Experience
AI Content Reviewer & Writer - Scale AI - Remote
(2023-01)
- Wrote evaluation prompts, including edge cases, to test model behavior at its limits
- Ranked AI-generated responses against rubrics covering helpfulness, accuracy, depth, and safety
- Ran side-by-side comparisons of multiple model outputs and documented the reasoning behind each choice
- Flagged hallucinations, reasoning gaps, and policy violations in model outputs
- Annotated data for intent, tone/sentiment, helpfulness, and safety compliance
- Wrote structured feedback identifying model weaknesses and areas for improvement
- Maintained consistency across datasets while following evolving project guidelines
Data Annotation Specialist (Independent) - Mercor.AI - Remote
(2024-01 - 2025-01)
- Produced written content across technology, AI, and education topics
- Edited drafts for clarity, tone consistency, and grammar
- Simplified technical concepts into plain-language writing for general audiences
Education
Bachelor of Science - Information Technology - University of Maryland Global Campus
Skills
AI Evaluation Specialist & Data Annotation Expert
