Estimating a climber's Center of Mass — fusing a sensorized wall and monocular video
Politecnico di Milano · Project ACCEPT (Adaptive Climbing for Cerebral Palsy)
Built a marker-less, out-of-lab method to track a climber's 3D Center of Mass during real climbs — replacing expensive Vicon-style motion-capture rigs with a single camera and a sensorized climbing wall.
The system fuses two data streams: a neural network (MediaPipe) extracts 2D pose from monocular video, while tri-axial load cells capture the 3D force vector at each grip. A geometric model reconstructs the Center of Mass; force data fills the depth coordinate the camera cannot see.
Validated against a Vicon optoelectronic system. Final accuracy: ~1.5 cm on x/y, ~5 cm on z — competitive with lab-grade systems, usable in real gyms with children in rehabilitation.