Post-Doctorant Editing And Conditional Generation With Text-To-Video Generation Models H/F - INRIA

A propos d'Inria Inria est l'institut national de recherche dédié aux sciences et technologies du numérique. Il emploie 2600 personnes. Ses 215 équipes-projets agiles, en général communes avec des partenaires académiques, impliquent plus de 3900 scientifiques pour relever les défis du numérique, souvent à l'interface d'autres disciplines. L'institut fait appel à de nombreux talents dans plus d'une quarantaine de métiers différents. 900 personnels d'appui à la recherche et à l'innovation contribuent à faire émerger et grandir des projets scientifiques ou entrepreneuriaux qui impactent le monde. Inria travaille avec de nombreuses entreprises et a accompagné la création de plus de 200 start-up. L'institut s'eorce ainsi de répondre aux enjeux de la transformation numérique de la science, de la société et de l'économie.
Post-Doctorant F/H Editing and Conditional Generation with Text-to-Video Generation Models Type de contrat : CDD Niveau de diplôme exigé : Thèse ou équivalent Fonction : Post-Doctorant A propos du centre ou de la direction fonctionnelleLe centre de recherche Inria de l'Université Grenoble Alpes regroupe un peu moins de 600 personnes réparties au sein de 27 équipes de recherche et 8 services support à la recherche.Son effectif est distribué sur 3 campus à Grenoble, en lien étroit avec les laboratoires et les établissements de recherche et d'enseignement supérieur (Université Grenoble Alpes, CNRS, CEA, INRAE, ...), mais aussi avec les acteurs économiques du territoire.Présent dans les domaines du calcul et grands systèmes distribués, logiciels sûrs et systèmes embarqués, la modélisation de l'environnement à différentes échelles et la science des données et intelligence artificielle, Inria Grenoble - Rhône-Alpes participe au meilleur niveau à la vie scientifique internationale par les résultats obtenus et les collaborations tant en Europe que dans le reste du monde.Contexte et atouts du posteTitre : Editing and Conditional Generation with Text-to-Video Generation ModelsSupervision : Dr Stéphane Lathuilière (INRIA-UGA)Funding : BPI contractContexte :Recent advancements in generative AI, and in particular diffusion models [1,2], have significantly enhanced the capabilities of text-to-video (T2V) models [3,4], allowing users to produce richly varied and imaginative scenes from natural language descriptions. These systems demonstrate strong scene diversity and flexibility, making them attractive for applications in entertainment, simulation, and human-computer interaction.However, a persistent limitation lies in their inability to enforce fine-grained conditioning and maintain strict consistency for specific visual elements. For example, while a T2V model can generate a person walking in a park, it struggles to ensure the persistent appearance of a specific object, character identity, or detailed attribute (such as a specific garment [5]) across complex poses and dynamic environmental interactions.In contrast, highly specialized image and video editing systems-such as those designed for virtual try-on [5], face swapping, or precise object insertion-excel at fine-grained conditioning on target individuals or objects. They can adapt elements to morphology, pose, and texture details with remarkable realism. Yet, these specialized approaches generally operate in isolation, lacking the scene diversity and broader contextual awareness that foundational T2V models offer.Bridging these two paradigms offers a powerful opportunity: to synthesize realistic, precisely controllable subjects and objects embedded within richly described, dynamic environments. To achieve this, novel alignment and editing techniques are required. Specifically, post-training with Reinforcement Learning (RL) presents a highly promising methodology to overcome these limitations. By leveraging RL during the post-training phase, foundation T2V models can be explicitly optimized to follow complex conditioning signals, enforce temporal consistency, and align with specific human-defined objectives for fine-grained editing tasks without sacrificing their generative diversity..Mission confiéeResearch Objectives :The primary mission of the Postdoctoral Research Fellow will be to advance the state-of-the-art in controllable and editable Text-to-Video (T2V) generation. The successful candidate will design, implement, and evaluate novel deep generative models and methodologies that address the current limitations of existing T2V systems. A core focus will be on achieving fine-grained conditional generation via post-training with Reinforcement Learning (RL), allowing users to specify complex temporal, spatial, and stylistic constraints, as well as enabling intuitive and high-fidelity post-generation editing of the video content. The research will aim to produce models that are not only photorealistic but also exhibit high semantic fidelity, temporal coherence, and practical usability in creative and industrial applications.Principales activités2. Main TasksThe Postdoctoral Research Fellow will be responsible for the following main tasks. They will engage in Model Design and Development by designing and implementing novel architectures (e.g., Diffusion Models, Transformers, VAEs) specifically tailored for high-resolution, temporally consistent, and controllable video generation. A key focus is to develop conditional generation techniques to guide the Text-to-Video process using various complex inputs beyond a simple text prompt, such as image references, motion skeletons, semantic masks, or detailed scene descriptions. They will extensively research Video Editing and Manipulation, developing methods for high-fidelity post-generation video editing, allowing for non-destructive modification of generated videos (e.g., object replacement, style transfer, background alteration) while maintaining strong temporal consistency. Furthermore, they will investigate in-context editing mechanisms that enable precise changes to specific segments or objects within a generated video based on new text or image prompts. A core part of the role is Addressing Key T2V Challenges. This includes tackling the fundamental challenge of temporal coherence and consistency, ensuring that generated videos do not suffer from "flickering" or object identity changes across frames, and developing strategies to improve semantic fidelity, resolving issues where models misinterpret complex text prompts. They will also explore methods for efficient training and inference to manage the significant computational cost associated with high-resolution, long-duration video generation, and address the difficulties of data scarcity and bias through techniques like data augmentation or cross-modal transfer learning. Finally, they will perform Evaluation and Benchmarking, establishing rigorous quantitative and qualitative metrics to assess the quality, editability, and controllability of the developed models. The fellow is expected to prioritize Dissemination and Collaboration, which involves documenting research findings and publishing high-quality papers in top-tier machine learning and computer vision venues, actively participating in departmental seminars, and contributing to collaborative projects.CompétencesCompétences techniques et niveau requis :We are seeking a motivated PhD candidate with a strong background in one or more the following areas :speech processing, computer vision, machine learning,solid programmming skillsinterest in connecting AI with human cognition Prior experience with LLM, SpeechLMs, RL algorithms, or robotic platforms is a plus, but not mandatoryLangues : AnglaisAvantagesRestauration subventionnéeTransports publics remboursés partiellementCongés: 7 semaines de congés annuels + 10 jours de RTT (base temps plein) + possibilité d'autorisations d'absence exceptionnelle (ex : enfants malades, déménagement)Possibilité de télétravail 90 jours/an fixes ou flottants et aménagement du temps de travailÉquipements professionnels à disposition (visioconférence, prêts de matériels informatiques, etc.)Prestations sociales, culturelles et sportives (Association de gestion des oeuvres sociales d'Inria)Accès à la formation professionnelleParticipation Protection Sociale Complémentaire sous conditionsRémunération2788€ gross salary / month