[Remote] Student Researcher (Seed - Infra - Inference) - 2026 Start (PhD)
Note: The job is a remote job and is open to candidates in USA. ByteDance is a leading technology company dedicated to pioneering advanced AI foundation models. They are seeking talented individuals for a PhD internship to actively contribute to their products and research, focusing on machine learning systems and distributed training.
Responsibilities
- Research and develop our machine learning systems, including heterogeneous computing architecture, management, scheduling, and monitoring
- Manage cross-layer optimization of system and AI algorithms and hardware for machine learning (GPU, ASIC)
- Implement both general purpose training framework features and model specific optimizations (e.g. LLM, diffusions)
- Improve efficiency and stability for extremely large scale distributed training jobs
Skills
- Currently in PhD program in distributed, parallel computing principles and know the recent advances in computing, storage, networking, and hardware technologies
- Familiar with machine learning algorithms, platforms and frameworks such as PyTorch and Jax
- Have a basic understanding of how GPU and/or ASIC works
- Expert in at least one or two programming languages in a Linux environment: C/C++, CUDA, Python
- Must obtain work authorization in country of employment at the time of hire, and maintain ongoing work authorization during employment
- GPU based high performance computing, RDMA high performance network (MPI, NCCL, ibverbs)
- Distributed training framework optimizations such as DeepSpeed, FSDP, Megatron, GSPMD
- AI compiler stacks such as torch.fx, XLA and MLIR
- Large scale data processing and parallel computing
- Experiences in designing and operating large scale systems in cloud computing or machine learning
- Experiences in in-depth CUDA programming and performance tuning (cutlass, triton)
Benefits
- Interns have day one access to health insurance
- Life insurance
- Wellbeing benefits and more
- Interns also receive 10 paid holidays per year
- Paid sick time (56 hours if hired in first half of year, 40 if hired in second half of year)
- Interns who are not working 100% remote may also be eligible for housing allowance
Company Overview
Company H1B Sponsorship