Introducing the Headphone Audio Quality Inventory (HAQI)

Hi guys, I am excited to share with you what I have been working on in the last year or so.

For my Master's thesis for my degree in Media Technology at TU Berlin I worked on the Headphone Audio Quality Inventory (HAQI).
I will keep it short here as I have attached my thesis (it's not too long, written as a paper) for anyone that is actually interested.

The HAQI aims at providing a solution to efficiently describe and measure the sound of headphones in a multidimensional perceptual way, not relying on physical measurements. It basically provides a questionnaire which can be used in listening tests for headphone evaluation, as well as a Structural Equation Model which is needed to transform the raw questionnaire data into scores and analyze the results.

Before I briefly outline the methodology, I will give a tl;dr of the current findings:

It was found that headphone (sound) perception has at least five latent dimensions:
Clarity which describes the perception of a detailed, clear sound, an accurate reproduction of the audio signal.
Spatial Precision (previously called "Spaciousness" in the thesis) describes the perception of precise localization of sound sources in the sound image.
Bass describes the perceived impact and punch of the bass reproduction.
Treble discomfort describes discomfort associated with perceived excessive or disturbing high-frequency content.
Wearing comfort describes exactly that, notably as some relationship with the overall sound impression.
Optionally also Openness which describes the extent to which the sound image is perceived as open.

With the first five dimensions the variance of liking of headphones could be explained by about 70 % and the overall acoustic impression by 77 %, with Clarity having the strongest positive relationship, Bass having a small positive relationship and Treble discomfort a negative one. Wearing comfort has a positive relationship with Liking and also the Overall acoustic impression.

These five/six latent dimensions cannot directly be retrieved in a questionnaire, but other questionnaire items which have to be answered in a listening test load on these factors. The full HAQI questionnaire is presented in this table:

HAQI_english.webp


Now a short description of the methodology:

In a first step, headphone sound-describing attributes from scientific literature and online forums were collected and structured, resulting in 127 attributes. These attributes were then discussed in an expert focus group consisting of nine German-speaking headphone/audio experts from science and industry (including Oratory1990 and other people known in the headphone community) and were eventually reduced to 68 items (CSV with all items with descriptions and poles in German and English attached below.)

In the next step, a listening experiment was conducted evaluating headphones using a questionnaire with these 68 items. The listening experiment was conducted (and still is being conducted) at nine different institutes in Europe, mostly Germany. Overall, 24 different headphone models ranging from super cheap to pricey, from consumer to pro audio, from very closed to very open etc. were used in the experiment with every institution using a subset of 7 of these headphones. In the listening experiment, participants were presented the headphones one after another, listening to one of three music snippets and evaluating the headphones based on the 68 items.

After that, the questionnaire data was analyzed using a exploratory factor analysis (EFA) to find the latent dimensions/factors which best describe the relationship of different items' responses. On the basis of the EFA results, a measurement model using confirmatory factor analysis (CFA) was constructed, adjusted and evaluated, finally resulting in the model shown above.
Basically what this model means is that you can calculate the (relative) manifestation of a perceptual dimension e.g. Bass by taking the ratings of the items sub-bass, punch and bass emphasis and "throwing them in the model" and a factor score can be computed which then can be compared to the score of another headphone.
There is of course much more to the analysis which is described in the attached thesis.

So what is the HAQI good for?

It could, for example, be used in the development or research with headphones to compare different prototypes/models with the respect to how their sound is perceived. A (blind) listening experiment would have to be set up employing different headphones (ideally including high and low anchors) and having a sufficient amount of participants (trained or untrained) take part. Using this model, conclusions could be drawn which model/prototype sounds "best" regarding certain perceptual dimensions, something a simple preference rating cannot explain.

There are, however, some limitations to the current model, which are described in detail in the thesis. Most importantly, this thesis is based on a limited set of data as there is still data coming in as I write. In (hopefully) the beginning of next year, a journal paper will be submitted and later published which will draw on the full data and will undergo review. I however wanted to share the current state as it is already very interesting and I am proud of what I achieved with the help of the experts, researchers and participants at the different institutions.

I am excited for further discussion on the topic with you guys!
 

Attachments

This is very cool. Thanks for posting. I haven't had time to dig into the full thesis yet but I can see this being very valuable in future research we have planned.

One question I have is about the terms in the spaciousness category. It seems to me that there is a lot of overlap between localizability and sense of direction, especially given that the poles are precise to imprecise. I'm struggling to imagine having a precise sense of direction for a source of sound but not being able to localize it. I'm also not sure I intuitively grasp what a precise or imprecise sense of distance refers to. Some headphones definitely sound nearer or further away, but I'm not sure which is more precise or imprecise.
 
Back
Top