Researchers on the Good Recording Project (University of Salford, UK) have just launched a new Web experiment investigating audio quality in Freesound samples.
The AudioBattle experiment presents participants with pairs of real samples taken from Freesound and asks them to rate which is better. The researchers hope to discover how well we can predict quality from audio features in recordings and in the metadata of sounds submitted to Freesound.
The experiment takes no more than 10 minutes and all participants are able to enter a prize draw once they finish. To take part in the experiment please visit: http://www.goodrecording.net/audiobattle/
Page crashed on sample #7.
Very interesting experiment, sadly I got to question nine and the next button doesn’t work (even after listening to each sound multiple times) I will try again tomorrow.
Interesting indeed, but a couple questions rise up: is there any point in comparing sounds which are qualitatively different? What’s the definition of ‘better’? (realistic? effective? pleasant?) Can something that runs on a qualitative scale be quantified? Which color is better? Mountain or beach for holidays? An so on…
Just as a clarification I’d like to point out that this experiment is being run by researchers at University of Salford and no members of the Freesound team are involved. I just say that because bug reporting should not be addressed to us (not this time! ;))
Regarding dobroide’s comment, I imagine that one possible outcome of the experiment will be a kind of ‘generic’ definition for ‘better’ quality (so to guess what sound properties do users consider when asked to compare sounds). It will be interesting how much agreement will they find in user responses and which sound characteristics are they analyzing.
Hi all, this is Iain from the Good Recording Project (the people behind the Audiobattle experiment). Thanks for the interest and feedback.
@sassyman, @luckylittleraven
Sorry to hear you had problems completing the test. It appears to be working fine at our end – can I ask what device/browser you were using?
@dobroide
Thanks for the comments. I’d agree that for professional recordings asking which is ‘better’ isn’t always of great value – as you suggest, in those cases ‘which do you prefer?’ or similar might be a more informative question.
Our project however focuses more on the audio quality of user generated (usually non-professional) content, like most of the samples on Freesound. In this content recording ‘errors’ (such as handling noise on a mic, poor balance between foreground and background noise, overloading the mic, etc) are commonly found and can be said to detract from the overall quality in a meaningful sense.
We hope that the current experiment will help to provide us with a better sense of what factors in everyday recordings are perceived as having the biggest effect on quality.
I’d say that dobroide’s point is a good one, and I don’t quite see how being professional necessarily has much to do with it; it’s more a matter of the intended use of a given sound, which isn’t indicated here. Some things like handling noise are unambiguous errors, but balance between foreground and background in particular isn’t; given that many FreeSound users like listening to ambient field recordings, deciding whether a recording with lots of general background is ‘better’ than another of a more isolated source will be fairly arbitrary in certain circumstances.
Anyway, good luck with your research… I’m sure peer review will be tougher than us lot 😉
Hi Iain, I dropped back to try the test again and saw you had replied. I was using an old desktop and IE. It could have been my dodgy rural internet connection as I completed the experiment today without a problem. Definately agree with Xinaes who says that intended use is quite an important factor in deciding which sound might be “better” but general audio errors like wind, movement and sound overpowering are quite obvious. Thanks and good luck with the project.