Staff Automated Model Evaluation & EvalOps Engineer

Opportunity Overview

Location: 100% Remote (United States)
Employment Type: Full-Time Direct-Hire via Accel DNA
Compensation Range: $180,000 – $215,000 Base + Equity
Hiring Agency: Accel DNA – Premier Frontier Data & AI Talent Partner

Establish continuous automated evaluation frameworks (EvalOps) assessing frontier LLMs, agent workflows, and retrieval pipelines for quality, regression, and safety.

Key Responsibilities:

• Architect automated CI/CD evaluation pipelines executing synthetic test cases across every prompt, model weight, or retrieval parameter change.
• Develop specialized LLM-as-a-judge rubrics, G-Eval metrics, and deterministic verification functions measuring faithfulness and accuracy.
• Construct enterprise domain-specific golden evaluation datasets with rigorous ground-truth curation.
• Build analytics dashboards visualizing model drift, hallucination frequency, latency degradation, and token expenditure.

What You Bring:

• 5+ years in data science or ML engineering with at least 2 years focused on LLM evaluation, benchmarking, and testing.
• Strong familiarity with modern eval tooling such as DeepEval, Ragas, Promptfoo, or Braintrust.
• Solid statistical grounding in inter-annotator agreement, bootstrap confidence intervals, and hypothesis testing.
• BS/MS in Data Science, Statistics, Computer Science, or equivalent.

How to Apply:

Interested in exploring this opportunity? Submit your profile or portfolio directly to contact@accel-dna.com or apply through https://accel-dna.com/contact-us/.