Phd Position F - M Privacy-Preserving Collaborative Learning Of Large Language Models Across Heterogeneous Learning Paradigms H/F - INRIA

A propos d'Inria

Inria est l'institut national de recherche dédié aux sciences et technologies du numérique. Il emploie 2600 personnes. Ses 215 équipes-projets agiles, en général communes avec des partenaires académiques, impliquent plus de 3900 scientifiques pour relever les défis du numérique, souvent à l'interface d'autres disciplines. L'institut fait appel à de nombreux talents dans plus d'une quarantaine de métiers différents. 900 personnels d'appui à la recherche et à l'innovation contribuent à faire émerger et grandir des projets scientifiques ou entrepreneuriaux qui impactent le monde. Inria travaille avec de nombreuses entreprises et a accompagné la création de plus de 200 start-up. L'institut s'eorce ainsi de répondre aux enjeux de la transformation numérique de la science, de la société et de l'économie.PhD Position F/M Privacy-Preserving Collaborative Learning of Large Language Models Across Heterogeneous Learning Paradigms
Le descriptif de l'offre ci-dessous est en Anglais
Type de contrat : CDD

Niveau de diplôme exigé : Bac +5 ou équivalent

Fonction : Doctorant

A propos du centre ou de la direction fonctionnelle

The Inria University of Lille centre, created in 2008, employs 360 people including 305 scientists in 15 research teams. Recognised for its strong involvement in the socio-economic development of the Hauts-De-France region, theInria University of Lille centre pursues a close relationship with large companies and SMEs. By promoting synergies between researchers and industrialists, Inria participates in the transfer of skills and expertise in digital technologies and provides access to the best European and international research for the benefit of innovation and companies, particularly in the region.For more than 10 years, theInria University of Lille centre has been located at the heart of Lille's university and scientific ecosystem, as well as at the heart of Frenchtech, with a technology showroom based on Avenue de Bretagne in Lille, on the EuraTechnologies site of economic excellence dedicated to information and communication technologies (ICT)

Contexte et atouts du poste

The widespread deployment of Large Language Models (LLMs) has given rise to diverse adaptation paradigms that accommodate varying computational and infrastructural constraints. In addition to traditional full fine-tuning, emerging methods such as prompt-tuning, parameter-efficient tuning like LoRA (Low-Rank Adaptation), and in-context learning for model adaptation with reduced computational resources or without access to model weights. These approaches open up new possibilities for collaborative learning in privacy-sensitive contexts, where multiple clients aim to improve LLM performance without exposing their raw data.

This PhD thesis will focus on designing privacy-preserving collaborative learning strategies for LLMs, starting in a homogeneous setting, where all participants rely on the same adaptation paradigm. This initial step will build a foundation for tackling the more ambitious and impactful goal of heterogeneous collaboration, where clients operate under different adaptation regimes due to diverse privacy, computational, or architectural constraints. A central challenge of the project is to reconcile contributions from such heterogeneous clients in a unified learning process, while ensuring rigorous privacy guarantees-most notably through differential privacy (DP), which provides strong theoretical protections against data leakage. The thesis will also address the trade-offs between model utility and privacy risk and propose novel mechanisms specifically tailored to this multi-paradigm collaborative learning scenario.

The PhD candidate will BE based at Inria Lille, within the MAGNET research team, and will BE co-supervised by M. Tommasi, Dr. Raouf Kerkouche (Inria Lille) and Dr. Cédric Gouy-Pailler (CEA Saclay). The research will benefit from a stimulating scientific environment, combining Inria's strong expertise in machine learning and artificial intelligence with the applied research focus of the CEA. This thesis is part of the REDEEM project, funded by the PEPR IA initiative (France 2030). IT offers a highly interdisciplinary environment bridging machine learning, natural language processing, and privacy-enhancing technologies, with opportunities for national and international collaboration.

Mission confiée

The person hired will carry out original research toward a PhD on Privacy-Preserving Collaborative Learning of Large Language Models Across Heterogeneous Learning Paradigms. The research will involve designing novel collaborative protocols, formalizing privacy guarantees, and evaluating theimpact of different learning paradigms on performance and privacy.

Principales activités

The candidate will get acquainted with the state of the art on privacy-preserving collaborative learning and adaptation of Large Language Models (LLMs), perform original research in close interaction with the thesis supervisors and other collaborators, and design collaborative learning strategies across heterogeneous adaptation paradigms such as full fine-tuning, prompt-tuning, and in-context learning. A key part of the work will involve developing and analyzing privacy-preserving mechanisms-particularly those based on differential privacy, which offers strong theoretical guarantees against data leakage. The candidate will evaluate these mechanisms in terms of their utility-privacy trade-offs, write scientific articles detailing the results, and present the work at top-tier international conferences and leadingpeer-reviewed journals in the areas of machine learning, privacy, and natural language processing.

Compétences
- Master's degree in computer science, artificial intelligence, machine learning, natural language
processing, applied mathematics, or a related field.
- Solid foundations in machine learning and natural language processing (NLP), including familiarity
with training and fine-tuning large language models (LLMs), as well as an understanding of recent
trends such as prompt-tuning and in-context learning.
- Experience or strong interest in collaborative learning.
- Familiarity with privacy-enhancing technologies, in particular differential privacy, is a plus.
- Strong programming skills, preferably in Python, and experience with machine learning frameworks
such as PyTorch or TensorFlow.
- Excellent oral and written English proficiency.

Please follow the guidelines described on for your application.

Avantages
- Subsidized meals
- Partial reimbursement of public transport costs
- Leave : 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
- Possibility of teleworking and flexible organization of working hours
- Professional equipment available (videoconferencing, loan of computer equipment, etc.)
- Social, cultural and sports events and activities
- Access to vocational training
- Social security coverage

Rémunération

2 200 € monthly gross salary