Alephalpha
AI Researcher - Pre-training (f/m/d)
Опубликовано
- География
- Berlin
Актуальность нужно проверить
Откройте публикацию в источнике и уточните, продолжается ли набор.
Источник проверяли 12 ноября
Доступ проверим при нажатии.
О вакансии
Вакансия «AI Researcher - Pre-training (f/m/d)» в компании Alephalpha. Место работы: Berlin.
Чем предстоит заниматься
- Your Responsibilities
- Recipe & Architecture Experiments: Design and run ablations on core elements of the training recipe (optimizers, schedules, initialization), and implement PyTorch-based architectural changes. Evaluate their effect on convergence, stability, and training efficiency.
- Scaling Experiments: Run and analyze small-scale proxy experiments, and contribute to the hyperparameter scaling laws and scale-up methods we use to predict behavior at multi-thousand-GPU scale.
- Run Monitoring & Debugging: Help monitor large-scale training runs and investigate convergence issues such as loss spikes and divergence. Work with the team to track down distributed training failures.
- Research to Practice: Follow the literature, reproduce promising results at small scale, and bring ideas worth testing to the team.
- Cross-Team Collaboration: Work with the Compute Performance, Data, Evaluation, and Post-Training teams to understand how hardware constraints and the rest of the model lifecycle shape pre-training decisions.
- Core Qualifications
- You are proficient in Python and have hands-on experience with PyTorch.
- You have a strong foundation in machine learning, shown through research (e.g., a PhD, a strong MS'c thesis, or publications), open-source contributions, or substantial professional or substantial personal projects.
- You have a solid mathematical background (linear algebra, probability, optimization) and enjoy reasoning formally about why models train the way they do.
- You understand how transformers work and have pre-trained LLMs from scratch yourself, even at small scale.
- You design experiments carefully, report noisy and negative results honestly, and can explain clearly what your evidence does and does not show.
- You have a track record of picking up unfamiliar technical areas quickly and going deep, for example by reimplementing papers, ramping up on a large new codebase, or debugging a problem well outside your comfort zone.
- You write clean, maintainable code and care about reproducible experiments.
- You communicate clearly, raise problems early, and actively seek out and act on feedback.
Что нужно для работы
- Preferred Qualifications
- Будет плюсом: Distributed Training: Experience training models on multi-GPU or multi-node setups, or familiarity with distributed frameworks (e.g., Megatron-LM, DeepSpeed, torchtitan) and parallelism strategies (data, tensor, pipeline, expert).
- Будет плюсом: Training Dynamics: Familiarity with scaling laws, hyperparameter transfer (e.g., µP), or recent work on optimizers and training stability.
- Будет плюсом: Efficient Architectures: Exposure to Mixture-of-Experts, efficient attention variants, or other inference-efficient designs.
- Будет плюсом: Research Output: Publications at top-tier venues (NeurIPS, ICML, ICLR) or contributions to open-source ML training libraries.
- Будет плюсом: Systems Curiosity: Low-level kernel optimization is not required, but we highly value a strong curiosity about the hardware and systems constraints that shape scale.
- Будет плюсом: Find more English Speaking Jobs in Germany on Arbeitnow
Источник: Arbeitnow. Условия и актуальность вакансии подтверждает работодатель.
Подходит вакансия?
Перейдите к исходному объявлению или контакту работодателя.
Доступ проверим при нажатии.