LLM Pre-training & Distributed Engineer (AI Infrastructure)

<p><span data-sheets-root="1">We are seeking a highly skilled LLM Pre-training &amp; Distributed Systems Engineer. This role is essential for orchestrating large-scale machine learning training runs and optimizing&nbsp; distributed infrastructure. The ideal candidate will have a deep understanding of GPU clusters and extensive experience in system engineering to ensure efficient and reliable training processes.</span></p> <p><strong>Responsibilities:</strong></p> <ul> <li>Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.</li> <li>Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.</li> <li>Automate checkpointing and failure recovery during month-long training runs.</li> </ul> <p><strong>Required Skills:</strong></p> <ul> <li>Deep expertise in 3D parallelism (Data, Tensor, Pipeline).</li> <li>Experience managing SLURM or Kubernetes-based GPU clusters.</li> <li>Strong systems engineering background (C++, CUDA, Python).</li> </ul> <p>&nbsp;</p>