Today, touch screens more and more replace the physical buttons, knobs and sliders of audio mixing devices, e.g. a groovebox.

However, tactile feedback is often missing. Adding this feedback, new innovative possibilities arise.


In this research study, a system was developed and evaluated, that uses audio driven vibrations for intuitive identification of music instruments.

SweetSpotter GUI



Touch screens allow the interface elements to be easily arranged to fit the user’s needs; however, if space and time constraints prevent the interface elements from being labeled, then the performer must remember all the connections between audio signals and mixer channels.

Alternatively, he might identify a specific audio signal by watching its VU meter or by prelistening to a channel using headphones. The latter is not very feasible as the auditory modality is already in use, e.g. during a live musical setup.

To use touch screens for traditional audio mixing consoles is not very common yet. One reason is the ease of use and the intuitive tactile feedback that hardware faders, knobs and buttons provide.

Adding tactile feedback to touch screens will largely improve usability. In addition touch screens can reproduce innovative extra tactile information.

Additional vibrations at the fingertip could be generated from the audio signal. This would improve the recognition of an instrument and help to avoid confusion of different audio channels.

SweetSpotter GUI


Tactile feedback was reproduced in an experiment using an electro dynamic vibration actuator coupled with a touch sensitive device. Alternatively, e.g. electro tactile feedback could be used.

A groovebox like interface was implemented. Participants had to perform a mixing task using different predefined audio signals. When the finger of the participant came in contact with a button or slider, tactile feedback for the respective audio channel was rendered in real time using Pure Data.

  SweetSpotter GUI  

Several design parameters for the generation of tactile feedback have been investigated. The results indicate that different music instruments can be distinguished if tactile feedback is rendered from the audio signal in an appropriate way. This helps to improve recognition of an audio source that is assigned e.g. to a specific mixing channel.

Applying this knowledge, the use of touch screens in audio applications becomes more intuitive. Even conventional mixing consoles could benefit from additional audio driven tactile feedback.

Want to know more? A detailed discussion and experimental data can be found in the following publications.

AES

Merchel, S., Altinsoy, M.E., Stamm, M. (2012). Touch the Sound: Audio-Driven Tactile Feedback for Audio Mixing Applications, in Journal of the Audio Engineering Society, 60(1/2), pp. 47-53.

PDF logo Merchel, S., Altinsoy, M.E. & Stamm, M. (2011). Tactile Identification of Non-Percussive Music Instruments, in Proceedings of the EAA Forum Acusticum 2011, Aalborg, Denmark.