Prosus
Senior Data Engineer, Training Data Platform
Опубликовано
- География
- Amsterdam
- Уровень
- Senior
Доступ проверим при нажатии.
О вакансии
Вакансия «Senior Data Engineer, Training Data Platform» в компании Prosus. Место работы: Amsterdam.
Чем предстоит заниматься
- Build acquisition and processing pipelines for portfolio data and relevant public and licensed sources, including connectors and content extraction. Turn commerce records into ordered histories and link behaviour to outcomes without leaking future information.
- Develop the shared platform for inspecting sources, configuring processing and publishing training datasets. Operate distributed workflows, monitor reliability and cost, and implement access controls and retention requirements.
- Build quality checks for completeness, freshness, duplication, extraction accuracy and coverage. Combine automated checks with sampled review to investigate problems that aggregate metrics can miss.
- Build synthetic data pipelines supporting pre- and post-training, including instruction examples, preference data and agent trajectories. Implement generation, validation, scoring and filtering. Work with the research team to measure how the generated data affects model performance, and manage generation throughput and cost.
- Build agents that turn catalogue sources into training datasets: propose tasks based on available data and target model capabilities, configure processing and generation, curate outputs and investigate quality failures. Keep agent actions traceable and validate proposed changes before dataset release.
Что нужно для работы
- At least five years of relevant data engineering experience, including ownership of substantial production pipelines.
- Strong Python and SQL, with experience writing tested, maintainable code.
- Hands-on experience with distributed processing using Spark, Ray, Beam or a comparable framework.
- Experience with cloud storage, workflow orchestration and operating pipelines through source changes, failures and backfills.
- Practical knowledge of data quality, versioning and lineage, and the ability to translate evolving research requirements into reliable implementations.
- Preferred qualifications
- Будет плюсом: Experience with LLM datasets, commerce data, content extraction, synthetic data generation or agents for data tasks.
- Будет плюсом: Experience with AWS, Databricks or EMR.
- Будет плюсом: Strong distributed-data engineering experience is welcome without prior experience on an LLM team.
Что предлагает компания
- Ownership of the platform supplying LCM's training and evaluation data, with direct collaboration with researchers and ML engineers.
- Colleagues who have released Hugging Face models with more than 100 million downloads, authored papers at NeurIPS, created benchmarks and built multiple production AI systems.
- Current ML frameworks and unlimited access to coding assistants.
- A base at our Amsterdam office and AI House, with relocation support. We also consider remote arrangements for candidates with particularly strong experience relevant to the role.
- At Prosus, your colleagues include 60,000 AI agents… and your job is to figure out what that actually means. We're one of the largest operators of agentic AI on the planet, inside a $100B+ portfolio across 100+ companies and two billion people. The humans here own what AI can't do alone: judgment, creativity, and the call on what gets built next.
- Own the impact. Work the ecosystem. Earn your growth. Build with backing.
- We respect the dignity and human rights of individuals and communities wherever we operate in the world. Building an inclusive workplace where everyone feels welcome and can thrive is critical for us. We provide access to education, which helps everyone understand the important role they play and the positive impact they can have.
Источник: Arbeitnow. Условия и актуальность вакансии подтверждает работодатель.
Подходит вакансия?
Перейдите к исходному объявлению или контакту работодателя.
Доступ проверим при нажатии.