Inference Optimization Engineer

San Francisco · Full-time · Posted Oct 8, 2026

At a glance

Hedra is the platform, models, and infrastructure for visual intelligence.
We build models and systems that push the frontier of visual intelligence, along with the infrastructure required to make those models fast, efficient, reliable, and accessible at scale.
We’re a small, highly technical team in San Francisco, backed by a16z and other leading investors. Researchers and engineers at Hedra work closely across boundaries, own problems end to end, and have significant influence over both what we build and how we build it.

The Inference Optimization Engineer will work alongside the research team to make state-of-the-art visual models fast and efficient at inference time.
This role involves working at the boundary between research and systems to optimize model architectures, runtimes, kernels, and distributed execution on modern hardware.

Work directly with research scientists and engineers to make new visual models fast and efficient at inference time.
Profile model architectures and workloads to understand bottlenecks across compute, memory, communication, and model execution.
Develop and implement new approaches to improving inference latency, throughput, memory efficiency, and GPU utilization.
Explore algorithmic optimizations including quantization, sparsity, caching, compilation, attention optimizations, and alternative execution strategies.
Build or optimize GPU kernels using CUDA, Triton, or similar technologies.
Optimize model execution across single-GPU, multi-GPU, and multi-node environments.
Reason about the interaction between model architecture and hardware, and work with researchers on architectural changes.
Investigate communication, memory movement, parallelism, and distributed execution strategies for large visual models.
Build rigorous benchmarking, profiling, and performance-regression infrastructure.
Evaluate new inference runtimes, compilers, frameworks, optimization techniques, and accelerator hardware.

Deep technical ability in efficient ML inference, ML systems, GPU computing, or adjacent research.
Strong understanding of how modern deep learning models execute on hardware.
Experience profiling ML workloads, identifying bottlenecks, and driving measurable performance improvements.
Strong programming fundamentals in Python, C++, or another systems-oriented language.
Experience with some combination of PyTorch, CUDA, Triton, TensorRT, vLLM, SGLang, or comparable technologies.
Ability to reason across abstraction layers.
Strong intuition for performance tradeoffs across latency, throughput, memory, numerical precision, model quality, and complexity.
Comfort working on ambiguous problems.
Ability to communicate technical ideas clearly and collaborate closely with research scientists and engineers.

Experience optimizing large generative, multimodal, vision, or video models.
CUDA, Triton, CUTLASS, or other GPU kernel development.
Deep knowledge of GPU architecture, memory hierarchy, and hardware-aware optimization.
Experience with attention optimization, kernel fusion, memory-efficient execution, or custom operators.
Model compilation or graph optimization experience.
Quantization, sparsity, caching, speculative execution, or other efficient inference techniques.
Multi-GPU or multi-node model execution, including tensor, pipeline, sequence, or other forms of parallelism.
Experience with profiling tools such as Nsight Systems or Nsight Compute.