Senior Real-Time Semantic Cache & Low-Latency LLM Platform Engineer

Opportunity Overview

Location: 100% Remote (United States)
Employment Type: Full-Time Direct-Hire via Accel DNA
Compensation Range: $170,000 – $205,000 Base + Bonus
Hiring Agency: Accel DNA – Premier Frontier Data & AI Talent Partner

Optimize enterprise LLM gateway architectures, deploying sub-millisecond semantic similarity caching, prefix caching, and intelligent token throttling.

Key Responsibilities:

• Build and scale distributed semantic caching layers using Redis, DragonflyDB, and vector proximity clustering to intercept redundant LLM calls.
• Configure prompt prefix caching and KV-cache reuse strategies on vLLM and TensorRT-LLM inference engines to maximize GPU throughput.
• Implement multi-tenant API routing gateways with dynamic fallback, circuit breaking, and latency-optimized model switching.
• Deliver significant operational cost reductions by achieving high cache hit rates on frequent enterprise inquiries.

What You Bring:

• 5+ years in high-throughput systems engineering and data platform development in Go, Rust, or Python.
• Demonstrated experience with caching technologies, vector search mechanisms, and proxy servers (Envoy, Nginx).
• Familiarity with foundation model inference mechanics (KV cache, continuous batching, chunked prefill).
• BS/MS in Computer Science or related technical discipline.

How to Apply:

Interested in exploring this opportunity? Submit your profile or portfolio directly to contact@accel-dna.com or apply through https://accel-dna.com/contact-us/.