We approached Imogen Heap with the idea, given the level of access she gives her fans in her creative practice, and her love of pushing boundaries in sound and technology. As hoped, Imogen was intrigued and asked us to carry out some audience testing of her latest video, You Know Where To Find Me, during Belfast Music Week. Below are the results, along with Imogen’s thoughts on it all - let us know yours in the comments below!
Bearing in mind that the exercise wasn’t carried out under lab conditions, we got a pretty consistent result from participants. Our sample group of around 20 included males and females of different age ranges, fans and industry professionals.
For any of you new to the Sensum emotional response platform, it adds to traditional feedback (via questionnaires) with Galvanic Skin Response (GSR) sensors to measure people’s skin conductance (sweat levels), which tells us their level of arousal in response to a piece of audio/video. Any increase or decrease in arousal could be due to positive or negative responses; excitement or fear, relaxation or boredom, and so on, so we used post-viewing questions and the Plutchik Wheel of Emotions to further investigate people’s responses.
For the You Know Where To Find Me video, we saw in most people’s emotional response a gradual increase in skin resistance - i.e. decrease in arousal - as the video progressed, with spikes of arousal at points where something changed (key changes, for example, or the re-introduction of vocals after an instrumental section). In the screen grab below you can see the early key change triggered an increase in arousal/sweat levels which shows as a decrease in resistance on the graph. For our upcoming beta launch we’re going to reverse the direction of the graph to make it more intuitive.
Post-viewing, most people chose ‘serenity’ on the Plutchik Wheel to describe their feeling - i.e. the decrease in arousal we were capturing was a result of them becoming more chilled out as the track/video progressed. When we shared this with Imogen, she replied, “Interesting results from Sensum regarding my song. Would be very curious to see the difference in results after having told the story behind the video to the song.” Indeed, two people told us that for them the video actually detracted from their enjoyment of the music - but when the background to the video was later explained, they became more interested.
The screen grab below shows a listener’s gradually increasing resistance - decrease in arousal - over the length of the track; this was a typical response as noted.
We also tested people’s emotional response to some of Imogen’s other videos, for comparison. The 'serenity’ response came back again for Canvas, 'surprise’ for First Train Home. Most people either liked the track slightly more than the video or liked both equally (aside from the two already mentioned who found the video a negative distraction from an enjoyable track), with a most common score of 8/10 across our sample group for both video and audio.
However, fascinating as it is to explore people’s emotional response to music, we’re especially interested in the idea of emotional response as a trigger. The Sensum platform was born out of a multi-disciplinary project called Biosuite, which the Sensum team ran in conjunction with the Sonic Arts Research Centre at Queen’s University, Belfast. As part of this project, which combined film production, music composition, environmental art, technology, and engineering to research audience interactivity and immersion in the audiovisual experience, an interactive horror short Unsound was created, where the audience’s aggregate emotional response triggered changes in the soundscape, camera angles and so on. (Read this New Scientist feature about it.)
Imogen’s “The Gloves” project uses gestural, motion data capture technology both as instrument and controller, so she was greatly interested in the potential of Sensum for real-time musical creation. She told us, “This has sparked my imagination to wonder what music I could write/play live that would be influenced, directed or conducted by the audience response in real time. Would love to see a film created live in this way too as I believe Sensum have been experimenting with.
"Imagining personal computers of the future via constant background emotional response reading and identifying their owner’s musical tastes so well as to be able to make on the spot music to accompany any mood or task. To inspire, calm down, excite, motivate. A whole new genre or area of music. Real computer music!”
A massive thank-you to Imogen for getting involved, and to all who took part in our testing. Thanks also to the staff at Harlem Cafe Belfast and the Belfast Music Week team for letting us do some guerilla testing during an industry event. See photos at www.facebook.com/sensumco, and find out more about the platform at www.sensum.co.
Are you a musician, music industry professional or music fan? What do you think of this technology and how it might change the craft, or our experience of music?