Hand Gesture Control of Sound: a digital musical interface

Abstract

This thesis reports on the development of a hand gesture driven musical instrument. Using the Nintendo Wii remote controller in combination with IR-LEDs attached to a glove, users can draw gestures in the air. The system interprets these motions in two manners. First by classifying a repertoire of analytical control gestures. Second by deriving higher order motional features, also named as holistic control gestures. For the former a recognition performance of above 99% is reached with a single training example. For the latter, a music-motion study is conducted on listener associations between musical changes and hand motions. The results indicate that many motional features are significantly affected by many musical parameters. This provides essential knowledge for musical mappings utilizing the holistic gestures, which is investigated by a proof-of-concept prototype of the musical instrument.