High-fiving with the machine: synthesising reactive motion from human input with interaction contact labels

Human motion synthesis has advanced significantly in domains such as text-to-motion and gesture generation, yet synthesising realistic interaction motion between multiple characters remains relatively underexplored, particularly in dynamic and user-driven environments like virtual reality (VR). Existing approaches rely on constructing sequences across a set number of frames, inherently limiting their usability in real-time and open-ended applications such as virtual reality. In this work, we present a novel latent diffusion-based framework for real-time interaction motion synthesis. Unlike prior models that operate on fixed-length sequences, our system continuously adapts to user behaviour, generating contextually appropriate and temporally coherent response motions. We introduce Interaction Contact Labels, a generalisation of ground contact annotations, to capture physical interdependencies between interacting characters and integrate them into the training objective to improve realism. Our approach enables open-ended, responsive character interactions suitable for live applications in VR, AR, and robotics. Further to this, we demonstrate a real-time implementation of the proposed method, which utilises CycleGANs to combine webcam data with high-quality motion capture data, allowing for easy end-user set-up.