Machine Listening as Sonification. András Blazsek's study was published on Organised Sound journal. The full text is available on the website of the Cambridge University Press.
"Machine listening takes place through sonification. It transforms sound into data that it analyses and rewrites again as sound, translating inaudible dimensions of the audible into the sonic realm. Conventional discourses around machine listening, however, do not discuss it as such; instead, they focus on it as a metaphor for human listening. By asking about the degrees by which they are separated or connected, this article analyses the relationship between human and machine listening.
French composer Pierre Schaeffer’s explanation of listening reveals a practice that shifts around different intuitive modalities; he says, ‘I understand [je comprends] what I was aiming to listen to [mon écoute], thanks to what I chose to hear [entendre]’ (Schaeffer Reference Schaeffer and North1966: 74). Additionally, he also defined raw perception of sound as ‘to perceive aurally’ (ouïr) (83). The separate but causal modes are also interchangeable: we first hear, then aim to listen by understanding the sound. For Schaeffer, the practice of listening is tangential to a series of relations, ‘I can understand the exact cause of what I have heard by connecting it with other perceptions or by means of a more or less complex series of deductions’; sometimes the lack defines this relationship or the disruption causes something else (78). Typically, people present machine learning as a technical process that mimics human behaviour; therefore, the question is, to what extent machine listening is a reproduction of our hearing process, and what the differences and similarities reveal.
The association between machine and human listening distracts attention away from the non-human kinds of relations among variables that develop in the machine as it listens in the way that only a machine can. These kinds of relationality are typically reduced in standard machine listening discourse as ‘interpretation’ and ‘understanding’, both of which separate objects to create mutually limiting relations of meaning-making between them. Thinking about machine listening as sonification permits an analysis that allows it to be examined as a non-human process that humans are using to create epistemological modes that are more-than-human, and aesthetic modes that are more-than-music.
Calling the process ‘listening’ already limits the machine’s process in defining the engagement that takes place between data and the analysis process. The discourse of sonification asks the human listener to understand the relations between values and sound. I argue that machine listening is a similar relational engagement between sonic data – samples, sonograms and scalograms in multidirectional translation processes that operate in hand with the intuition of the human listener to understand (comprendre) it as different. Hearing a perfect mimicry of someone’s voice, and not recognising it as different, proves our deafness to the more-than-human epistemic turn."
