Thèse Articuler les Dimension Physiques et Temporelles Combiner Jumeau Numérique et Réalité Mixte pour l'Analyse de Données et l'Aide à la Décision dans les Bâtiments H/F - Doctorat.Gouv.Fr

Établissement : Université de Toulouse École doctorale : EDMITT - Ecole Doctorale Mathématiques, Informatique et Télécommunications de Toulouse Laboratoire de recherche : IRIT : Institut de Recherche en Informatique de Toulouse Direction de la thèse : Emmanuel DUBOIS ORCID 0000000204791036 Début de la thèse : 2027-10-01 Date limite de candidature : 2026-11-23T23:59:59 Ce projet de thèse explore comment la réalité mixte peut permettre l'intégration des jumeaux numériques des bâtiments afin de rendre les données environnementales et énergétiques plus accessibles et exploitables par un large éventail d'acteurs.
Dans le contexte des changements climatiques et de la transition énergétique, les jumeaux numériques sont de plus en plus utilisés pour l'analyse des données de comportements énergétiques dans les bâtiments. Ils agrègent des données provenant de multiples sources, telles que des relevés historiques, des capteurs fonctionnant en temps réel ou encore des modèles de simulation capables de prédire les consommations futures. Ces systèmes établissent un lien entre le bâtiment physique et sa représentation numérique, généralement sous la forme de maquettes 3D ou de plans 2D. Toutefois, les plateformes actuelles sont principalement conçues pour des utilisateurs experts qui analysent ces données depuis des postes de travail classiques. Cette approche crée une dissociation spatiale et temporelle entre l'analyse des données et l'environnement physique auquel elles se rapportent.

Le projet vise à dépasser cette limite en s'appuyant sur la réalité mixte et la visualisation située, afin de permettre aux utilisateurs d'accéder directement aux informations et de les interpréter au sein même des bâtiments concernés.

L'objectif final est de recontextualiser les données des bâtiments pour les transformer en connaissances directement mobilisables par des publics non spécialistes, tels que les occupants, les techniciens de maintenance ou les décideurs en charge des politiques de rénovation et d'aménagement. En offrant un accès direct et contextualisé à des informations sur les consommations énergétiques et l'utilisation des espaces, le projet cherche à favoriser une prise de décision mieux informée ainsi qu'une meilleure appropriation des enjeux liés à la performance des bâtiments.

Pour atteindre ces objectifs, trois défis majeurs en interaction humain-machine devront être relevés. Le premier concerne la conception de techniques efficaces de visualisation et de manipulation de données multicouches à travers des interactions spatiales et gestuelles, tout en limitant la charge cognitive et la fatigue physique des utilisateurs. Le deuxième porte sur le développement de mécanismes d'interaction capables d'aider les utilisateurs à établir des corrélations pertinentes entre des sources de données hétérogènes, afin de faciliter la compréhension des dynamiques complexes du bâtiment sans nécessiter d'expertise préalable. Enfin, le troisième défi concerne la découvrabilité des données et des interactions disponibles dans des environnements situés. Le système devra fournir un guidage intuitif et proactif permettant aux utilisateurs d'identifier rapidement les éléments interactifs, de comprendre les actions possibles et d'interpréter les informations affichées avec un minimum d'apprentissage.

À l'intersection de l'Interaction Humain-Machine, de la Réalité Mixte et des Jumeaux Numériques, cette thèse ambitionne ainsi de concevoir de nouvelles approches d'analyses situées capables de rapprocher les données complexes des bâtiments des processus réels de décision et d'action.
The contextual framework of this PhD project is grounded in the concept of situated information, whereby a data representation is spatially co-located with the physical referent to which it relates [Willett 17]. In this context, numerous studies have shown that exploring data within its geographical setting strengthens the connection between the spatial environment and digital information [Maquil 16], improves users' perception of the physical environment represented by the data, and enhances memory and proprioception [Taher 15]. Furthermore, mixed reality provides a particularly suitable paradigm for situated information, as it enables data visualizations to be spatially anchored in a physical environment while remaining dynamic and flexible.
This perspective is also aligned with theories of situated cognition, which posit that cognitive activity is not confined to internal mental processes but is distributed across individuals, artifacts, and their environment. Situated cognition further encompasses temporal, social, and activity-related dimensions, and the degree of situatedness of a visualization depends on how these dimensions are articulated and combined [Bressa 21]. The situated nature of information and its spatial embodiment also contribute to enhancing users' situation awareness [Endsley 03]. Consequently, situated data visualization in mixed reality serves not merely as a visual anchor but as an external cognitive resource that structures reasoning processes and reduces the cognitive effort required to map data onto elements of the surrounding space. These concepts have been extensively explored in domains such as education [Lucignano 18], but have received comparatively less attention in the context of digital twin data visualization.
Finally, this type of data exploration can also be supported through the use of interactive 3D physical urban models [Ens 21]. Whether representing scaled-down neighborhoods or real urban environments, physical elements such as buildings, streets, signs, and other urban objects can be associated with projected or augmented reality (AR) visualizations. These elements act as physical referents through which users can access associated data in a variety of contexts, including urban planning, energy consumption monitoring, and crisis management. Numerous studies have investigated different approaches for displaying data in such environments. More recently, a systematic study examined the impact of information placement around physical referents and its effects on user interaction and interpretation [Benkhelifa 25].

However, the data currently displayed in such systems are often limited in terms of quantity, diversity, and complementarity. Yet understanding how occupants behave and how their activities influence building performance, energy consumption, and ultimately environmental impact requires the integration of multiple interdependent and correlated data sources.

It is within this context that the sobOCampus project is being conducted in Toulouse. The project brings together energy specialists, computer scientists, and end users to create a true living laboratory for rethinking building energy management. By combining simulation models, real-world data, and collective intelligence, it promotes a scientific approach to smart buildings that is firmly grounded in the realities of university campuses. The project is centered on a 3,000 m² building located on the University of Toulouse campus, comprising classrooms, meeting rooms, and an experimental hall, together with an infrastructure capable of connecting a wide range of sensors and its associated 3D digital twin. This ecosystem enables the detection of anomalies, the analysis of building energy performance, the prediction and optimization of future energy consumption, and the visualization of building-related data.
On the other hand, the work of the DOMUS laboratory at the Université de Sherbrooke focuses, among other things, on the design of smart homes to support aging. Through projects such as NEARS-SAPA (funded by the Canadian Institutes of Health Research (CIHR)), sensor networks, digital twins, and human activity recognition models make it possible to augment the homes of older adults and to provide clinicians with clinical dashboards to support their decision-making around autonomy support. Mixed reality is used in several research sub-projects, allowing clinicians to be guided in the deployment of smart homes [Magri 26], enabling room recognition within a home in mixed reality for scene understanding [Georges 26], and providing graduated assistance with activities of daily living using mixed reality headsets [Spalla 26]. The proposed thesis project will complement these achievements by bringing digital twin data visualization directly into the homes, will be complementary to the work of [Magri 26], and will align with the laboratory's research objectives of making smart homes and cognitive assistive technologies more interactive and usable for both clinicians and residents.



This research project focuses on advancing the use of environmental and energy-related building data through the use of digital twins using mixed reality technologies. In the context of climate and energy transitions, building analysis increasingly relies on digital twins [Benkhelifa 25], which aggregate multi-source data originating from historical records, real-time sensor networks, and even simulation engines capable of predicting future energy consumption.
These systems establish a link between the physical environment, from which data are collected, and a digital replica, typically represented as 3D models or 2D plans. They are primarily designed for data analysis by specialists who visualize and interpret information from desktop-based environments. As a result, both the temporal and spatial dimensions of data analysis are separated from the physical environment to which the data refer.
However, the benefits of situated visualization have been widely demonstrated in the literature [Willett 17].

The main objective of this project is to exploit the capabilities of digital twins to recontextualize environmental and energy-related data, thereby making it actionable for non-expert stakeholders, including building occupants, maintenance personnel, and policymakers responsible for planning and renovation decisions. The project seeks to provide these users with direct access to insights derived from analyses of energy consumption and space usage through situated interactions, that is, within the buildings and living environments to which the data directly relate.
To achieve this, the project defines the following methodological objectives:

*** Designing interaction modalities for exploiting digital twins for in situ data visualization: Develop interfaces that enable seamless navigation across heterogeneous information sources (e.g., energy consumption data, people-flow data) and multiple temporal dimensions (historical records, real-time data streams, and simulated future projections).

*** Modeling spatio-temporal filters: Design mechanisms that allow users to segment and explore data according to defined time ranges, sensor categories, or specific spatial units (e.g., rooms, floors), while remaining in close proximity to the corresponding physical installations.

*** Integrating everyday objects as interaction media: Investigate how everyday objects can be leveraged to support intuitive interaction with and visualization of digital twin data in mixed reality environments.


Achieving these methodological objectives requires addressing three key research challenges in the field of human-computer interaction:

*** Data Layer Management:
The use of in-situ interfaces, particularly those relying on spatial and gestural interactions (mid-air interaction), requires the development of representation and manipulation techniques that keep cognitive load and physical fatigue within acceptable limits for extended use. A central challenge is therefore to determine how multiple layers of information can be effectively visualized and accessed without overwhelming users.

*** Supporting Correlation Building:
This challenge concerns the modeling of interaction mechanisms capable of assisting users in establishing meaningful relationships between multiple data layers. The goal is to structure transitions between different information dimensions, thereby facilitating the understanding of building dynamics and performance by non-expert users.

*** Managing the Discoverability of Data and Available Interactions:
Users are generally accustomed to interacting through the traditional mouse-screen-keyboard paradigm. In situated environments, they must be able to quickly identify and use available interaction resources without extensive training. Proactive and anticipatory feedback is therefore essential to help users discover interaction zones and their associated manipulation techniques, as well as data visualization areas and the information they contain. This challenge draws on principles from nudge theory to guide user attention and support effective interaction.

The proposed contributions will be evaluated through both qualitative and quantitative assessments. Qualitative evaluations will rely on established questionnaires such as SUS (System Usability Scale), NASA-TLX, and Borg scales, while quantitative measures will include error rates and performance indicators. Controlled experiments will be conducted in laboratory settings, and longer-term field studies may be planned in collaboration with the partners' two application domains.
The PhD project will build upon a comprehensive literature review of the use of digital twins in the context of building energy management and monitoring. This first phase will aim to identify existing human-computer interaction approaches, the challenges reported in the literature, and current research gaps. The review will be complemented by a user needs analysis involving stakeholders from the two application contexts considered in the PhD project (the Toulouse and Sherbrooke initiatives).
An iterative methodology combining ideation (brainstorming), prototyping, and evaluation phases will be employed to design and develop innovative interactive solutions addressing one or several of the three identified research challenges.
Finally, a conceptual framework will be progressively established to document the project's outcomes and to capitalize on the insights and lessons learned from these experimental investigations.

Compétences requises
- Master en IHM (HCI), VR, informatique ou autres domaines pertinents
- Expérience avec les technologies AR / VR (logiciels, matériels)
- Expérience en conception et prototypage en IHM
- Connaissances en graphismes 3D
- Compétences en programmation : C#, Unity
- Compétences de communication écrite et orale en anglais et en français
- Capacités d'analyse et de résolution de problèmes

Compétences souhaitées
- Expérience avec des moteurs de jeu (Unity, Unreal Engine) ou des frameworks 3D
- Connaissance des méthodes de conception et d'évaluation de l'expérience utilisateur
- Expérience préalable en recherche en IHM ou VR/AR/XR ou domaines connexes

Application link : https://edd-projets.utoulouse.fr/