Music in Motion

A real-time motion-responsive sound system that transforms human movement into sound.

Terra NYC STEM FairEngineeringMediaPipeIMUPython

Can real-time sensing technologies be combined to create a continuous, responsive system that transforms human movement into sound? This project integrates MediaPipe pose estimation with wearable IMUs to map full-body motion to timbre, volume, and spatial audio—supporting dance, wellness, and interactive performance with low-latency, intuitive feedback.

Explore the project

Demo video

Watch the Python apps in action with real-time motion-to-sound mapping (Vimeo).

Watch demo

Poster board

Summary of the system architecture, design iterations, and key learnings.

View poster (PDF)

Source code

Python apps: MediaPipe, IMU handling, sensor fusion, and real-time audio.

Open on GitHub