- 96
MuSS-Bits: Music-Sensory-Substitution Bits
We conceptualized and developed Music Sensory Substitution (MuSS) Bits to support users with hearing impairments in music making activities. MuSS-Bits are small, plug-and-play, sensor-display pairs that capture and process sound from different real-world audio sources (e.g., musical instrument), communicate wirelessly and provide a visual and vibrotactile representation of the audio signal in real-time.
Existing sensory substitution systems work for music-listening but are not developed for music-making as they do not take music-making specific requirements into account, such as free limb movement. Based on a comprehensive literature review, observational studies and interviews with deaf musicians and music teachers from deaf schools, we identified four primary requirements: (1) real-time feedback from real-world audio sources, (2) feedback customizability, (3) simple to use, and (4) free limb movement.
With digital fabrication facilities, we implemented a high-fidelity prototype of MuSS-Bits and deployed this prototype in a controlled and a field study in residential deaf school in Sri Lanka. The feedback from both deaf children and their music teacher has been very positive. We believe MuSS-Bits hold a lot of potentials to support music-making and will ultimately create a positive impact on the lives of deaf children providing a “rhythm to their lives”.
Existing sensory substitution systems work for music-listening but are not developed for music-making as they do not take music-making specific requirements into account, such as free limb movement. Based on a comprehensive literature review, observational studies and interviews with deaf musicians and music teachers from deaf schools, we identified four primary requirements: (1) real-time feedback from real-world audio sources, (2) feedback customizability, (3) simple to use, and (4) free limb movement.
With digital fabrication facilities, we implemented a high-fidelity prototype of MuSS-Bits and deployed this prototype in a controlled and a field study in residential deaf school in Sri Lanka. The feedback from both deaf children and their music teacher has been very positive. We believe MuSS-Bits hold a lot of potentials to support music-making and will ultimately create a positive impact on the lives of deaf children providing a “rhythm to their lives”.
