FAQ for CLIP2
This page contains some background information on the topics of lyric intelligibility, hearing loss and objective measures. We will also be updating it with answers to any challenge-specific questions that we receive.
What is Lyric Intelligibility?
Lyric intelligibility, as defined by Cadenza's sensory panel of hearing aid users, refers to "how clearly and effortlessly the words in the music can be heard". Fine et al. [1] reported that trained and amateur singers (including singing teachers) believe that the understanding of the sung message while hearing a song is very important for their enjoyment.
For the CLIP2 challenge, the intelligibility metric is the percentage of words in a sentence that a listener identified correctly. This makes it compatible with the definition in CLIP1 and the common definition of speech intelligibility.
How was Lyric Intelligibility measured with listeners?
We used Prolific to run lyric intelligibility tests. Listeners with normal hearing transcribed the words from song excerpts processed in various ways, e.g. as-is and passed through a hearing loss simulation.
Why are we using AI-generated music?
There is a lack of openly available music recording datasets with annotated lyrics, and the existing data is often limited in the number of tracks and genres. Therefore, we used AI-generated music to fill this gap. This allowed us to obtain a large number of songs with lyrics across a wide range of genres, from Blues and Jazz to Musical Theater.
Additionally, because both the lyrics and songs were generated by AI models, there was no risk of listener familiarity during the listening tests, making the intelligibility scores more objective.
Previous lyric intelligibility metrics
- CLIP1 results, where 27 systems from 22 team were submitted, presenting different approaches that often combined high-level embedding representations complemented with traditional signal features and perceptual metrics.
- Singing adapted STOI and vocal-specific features model, which combines the traditional STOI metric with vocal-specific features to account for the different singing styles.
- Lots of speech intelligibility metrics that might be adapted to singing.
Learning Resources
We have extensive resources that provide background information on many aspects: hearing loss, hearing aid processing, mixing/remixing, and perceptual testing. To see current discussions of the challenge, please join the Google group.
I'm unclear if my approach is within the rules
Please contact the Cadenza Team via cadenzachallengecontact@gmail.com with details of what you want to do so we can advise.
What data can I use?
You should be able to find the information on the Rules page.
How do I actually enter?
See the Submission page. Please make sure you have signed up to our Google group, as this will ensure you get updates about the challenge.
References
[1] Fine, P. A., & Ginsborg, J. (2014). "Making myself understood: perceived factors affecting the intelligibility of sung text," Frontiers in psychology, 5, 809.