# Senior Real-Time Semantic Cache & Low-Latency LLM Platform Engineer

- Link: https://accel-dna.com/jobs/senior-semantic-cache-low-latency-llm-platform-engineer/
- Published: 2026-10-28T21:04:00+00:00
- Author: Umar Mahmud

## Opportunity Overview

**Location:** 100% Remote (United States)
**Employment Type:** Full-Time Direct-
Hire via Accel DNA**Compensation Range:** $170,000 – $205,000 Base + Bonus**Hiring
Agency:** Accel DNA – Premier Frontier Data & AI Talent Partner

Optimize enterprise LLM gateway architectures, deploying sub-millisecond semantic
similarity caching, prefix caching, and intelligent token throttling.

## Key Responsibilities:

 * Build and scale distributed semantic caching layers using Redis, DragonflyDB,
   and vector proximity clustering to intercept redundant LLM calls.
 * Configure prompt prefix caching and KV-cache reuse strategies on vLLM and TensorRT-
   LLM inference engines to maximize GPU throughput.
 * Implement multi-tenant API routing gateways with dynamic fallback, circuit breaking,
   and latency-optimized model switching.
 * Deliver significant operational cost reductions by achieving high cache hit rates
   on frequent enterprise inquiries.

## What You Bring:

 * 5+ years in high-throughput systems engineering and data platform development
   in Go, Rust, or Python.
 * Demonstrated experience with caching technologies, vector search mechanisms, 
   and proxy servers (Envoy, Nginx).
 * Familiarity with foundation model inference mechanics (KV cache, continuous batching,
   chunked prefill).
 * BS/MS in Computer Science or related technical discipline.

## How to Apply:

Interested in exploring this opportunity? Submit your profile or portfolio directly
to [contact@accel-dna.com](https://accel-dna.com/jobs/senior-semantic-cache-low-latency-llm-platform-engineer/contact@accel-dna.com)
or apply through [https://accel-dna.com/contact-us/](https://accel-dna.com/contact-us/).