The Attentive Hearing Aid: visual selection of auditory sources

Abstract

We present the Attentive Hearing Aid, a system that uses eye input to amplify the audio of tagged sound sources in the environment. A multidisciplinary project, we use the latest technology to take advantage of the social phenomenon of turn-taking in human-human communication, and apply this in a new kind of assistive hearing device. Using hearing-impaired participants, we evaluated the use of eye input for switching between sound sources on a screen in terms of switch time and the recall of audiovisual material. We compared eye input to a control condition and two manual selection techniques: using a remote to point at the target on the screen, and using buttons to select the target. Results show that in terms of switch time, Eyes were 73% faster than Pointing and 58% faster than Buttons. In terms of recall, Eyes performed 80% better than Control, 54% better than Buttons, and 37% better than Pointing. In a post-evaluation user experience survey, participants rated Eyes highest in “easiest”, “most natural”, and “best overall” categories. We present the implications of this work as a new type of assistive hearing device, and also discuss how this system could benefit non-hearing-impaired individuals.