Staff vLLM & TensorRT-LLM High-Throughput Inference Engineer

Remote
USA
Posted 1 day ago

Opportunity Overview

Location: 100% Remote (United States)
Employment Type: Full-Time Direct-Hire via Accel DNA
Compensation Range: $220,000 – $275,000 Base + Performance Bonus
Hiring Company: Accel DNA – Enterprise Data, Analytics & AI Talent Solutions

Architect enterprise-grade, high-concurrency LLM serving engines focused on maximizing token-per-second generation while slashing operational GPU inference expenditures.

Key Responsibilities:

  • Deploy, tune, and maintain distributed model serving topologies using vLLM, TensorRT-LLM, and TGI.
  • Implement speculative decoding, continuous batching, and chunked prefill strategies for multi-tenant enterprise traffic.
  • Optimize GPU memory layout, FP8/INT4 quantization kernels (AWQ, GPTQ), and PagedAttention configurations.
  • Establish real-time telemetry tracking time-to-first-token (TTFT) and inter-token latency (ITL) under heavy synthetic loads.

What You Bring:

  • 5+ years in high-performance computing, systems engineering, or MLOps infrastructure.
  • Deep technical mastery of modern inference acceleration runtimes, CUDA kernel profiling, and Linux GPU drivers.
  • Experience deploying production AI models on Kubernetes with Ray Serve and Triton.
  • BS/MS in Computer Engineering, Computer Science, or equivalent practical experience.

About Accel DNA & How to Apply:

Accel DNA is North America’s specialized talent partner for Data Science, Generative AI, and Machine Learning. Headquartered at 30 Wall Street, New York, we connect top-tier technical practitioners with industry-leading enterprise teams.

Interested in exploring this opportunity? Submit your profile or portfolio directly to umar@accel-dna.com or apply through https://accel-dna.com/contact-us/.

Job Features

Job Category

AI/ML

Apply For This Job

loader

Submit Your Resume

Our team will get in touch within 24 working hours.
Sign Up for Email and Phone Calls