Machine Learning Engineer, Speech LLMs

 

You’d be an early member of a new speech LLM lab inside a profitable, bootstrapped company, with the job of training models that understand and generate speech natively.

 

About the company

A profitable Voice AI company based in San Francisco. It builds hardware and software that capture what professionals say and hear, then turn it into notes and usable information. The business has a large global user base and is now setting up a core SpeechLLM lab, which this person would join as a founding hire.

 

The role

You’ll build and train large-scale audio and speech models from the ground up. That could mean unified SpeechLLMs, advanced ASR, expressive TTS or generative audio architectures. One day you’re designing a new sequence modelling architecture, the next you’re debugging a distributed training cluster. The work runs across the whole stack, from signal processing and raw acoustic representations to foundation model training and optimisation for edge devices. You’ll own ambiguous problems and take them into production.

 

What we’re looking for

A proven track record of building and training large-scale audio or speech models. Deep expertise in PyTorch or JAX, including optimising large distributed training runs, managing GPU memory and fixing performance bottlenecks. Comfort moving between research and engineering, and a real interest in AI systems that handle speech natively.

 

Any of the following is a bonus:

  • Text LLM pretraining, instruction tuning or RLHF
  • Neural audio codecs for streamable, high-fidelity audio
  • Diffusion, flow matching or autoregressive models for speech and voice generation
  • RL techniques such as RLHF or GRPO to improve conversational cadence, steerability and alignment
  • Inference optimisation with serving frameworks like vLLM, TensorRT-LLM or SGLang
  • Large GPU clusters, distributed training with FSDP or DeepSpeed, and Kubernetes

 

Salary and benefits

  • Up to $270K, plus a performance bonus and equity.
  • Healthcare for you and your dependents, including dental and vision, with a generous employer subsidy.
  • 401(k) with company matching.
  • Unlimited PTO plus 13 paid holidays.

 

Logistics

Full-time, permanent. Based in San Francisco, hybrid, with a minimum of three days a week in the office.