Rock Thomas

Data-driven and detail-oriented Computer Science professional with extensive experience in AI model evaluation, training, and data annotation. Proven track record across platforms like Outlier and Mindrift in executing Reinforcement Learning from Human Feedback (RLHF), prompt engineering, and rigorous adversarial testing (red-teaming). Expert in establishing high Inter-Annotator Agreement (IAA) and refining complex technical datasets.

Adept at leveraging a strong background in software development and technical writing to optimize LLM performance, safety, and accuracy.

Freelance AI Trainer / Evaluation Specialist at Outlier (2022-01 – 2024-12)

• Conducted adversarial testing and red-teaming protocols to identify vulnerabilities, hallucinations, and safety violations in enterprise-level LLMs.
• Authored high-complexity prompts across technical, mathematical, and logical domains to evaluate and score multi-turn model responses.
• Performed granular data annotation, constraint-following analysis, and attribute tagging, consistently maintaining top-tier quality metrics.
• Collaborated asynchronously with global teams to align annotation outputs with strict project rubrics and quality guidelines.

AI Writer & Trainer at Mindrift (2021-01 – 2022-12)

• Evaluated and optimized generative AI text outputs for factual accuracy, linguistic coherence, and tone alignment.
• Created high-quality gold-standard datasets used for the fine-tuning and supervised training of LLMs.
• Synthesized complex technical concepts into clear, structured prompt-response pairs to improve model reasoning capabilities.

AI Trainer at Echo Labs (2020-01 – 2022-12)

• Contributed to high-volume transcription, linguistic categorization, and data auditing projects to train specialized audio-to-text AI models.
• Reviewed automated outputs against strict ground-truth data to identify systematic errors and improve model confidence scores.
• Maintained rigorous data security and confidentiality protocols while handling proprietary training sets.

Bachelor of Science in Computer Science – Arizona State University (2022-05)