Technical Description
Alternative approaches might involve generic 3D avatars, chatbot interventions, or mental imagery. However, our immersive solution uses generative diffusion models to enable a realistic and multisensory dialogue with one’s past self. Based on substantial research in psychology and neuroscience, involving different senses, embodiment, and a realistic child’s depiction, it is thought to enhance benefits compared to alternatives.
Where traditional Deepfake techniques only replace the face and leave the hair, head shape, and facial expression unchanged, our approach completely regenerates the head from scratch based on the input image as well as text cues. The result is a hyperrealistic replica, offering full control over its facial expression. This is achieved by combining a multitude of open-source AI models into an customizable and easy-to-use processing pipeline.
Though video-based solutions offer linear results compared to more interactive alternatives such as 3D avatars, they yield much more realistic and compelling results. This, combined with our tool for providing tactile feedback when comforting the seen child has yielded powerful results as experimentally evaluated.
Our work is unique in combining a video-based solution with touch, perspective swaps, and psychophysiological tracking within VR. As one of the first teams applying deep fakes for mental health, our work is based on substantial interdisciplinary research, and a strong commitment to rigorous scientific evaluation. The project blends artistic sensibility with quantitative neuropsychological tools to create impactful, accessible mental health interventions.