Leiden Mobile Mocap Lab

Understanding how people move, sign, and gesture in real-life contexts is central to studying human communication — and increasingly important as AI systems move toward multimodal understanding, integrating language, vision, and movement. Yet traditional motion capture systems are often bulky, expensive, and confined to the lab.

The Leiden Mobile Mocap Lab has taken its first concrete steps. The project’s goal is to create a portable and flexible motion capture setup that makes it easier to study sign language, gesture, and other forms of embodied, multimodal communication and interaction at Leiden University.

Over the past months, we’ve been working on the technical setup. The core equipment has now arrived: a powerful laptop equipped with a strong GPU and 64GB of RAM for real-time video processing and local AI foundation model execution, and four 4K webcameras with embedded pose estimation models for recording from multiple angles and generating data for modelling 3D motion. Together, these form the basis of a compact motion capture kit that can be used both in the lab and in the field. We want to challenge the LUdev teams to create some inspiring case studies, and provide Makerspace style documentation for the kit.

The goal of the LUdev project is to activate the Mobile Mocap lab by creating a sign language experiment, as well as a creative gesture or motion-based installation as example use cases, with sufficient makerspace online documentation of the underlying components to inspire others to leverage the lab for their own projects. Obviously more demo case studies and community building activities can be added, and ideally we close the project with some form of workshop event.

Examples of other gesture and motion analysis projects and use cases (not created with the kit): - Pose estimation for sign language: https://dhq.digitalhumanities.org/vol/15/1/000510/000510.html - Creative application: https://zoebreed.com/pure-dance