Do we hear what we measure? Do we measure what we hear?

[rebbiputzmaker]

As we travel down this Noble Path of measurement to better understand the quality of our audio equipment, do we actually hear what we are measuring? Or measure what we hear?

Being subjective seems to be frowned upon but, are we taking these measurements and then
(maybe unwittingly) carrying them over to listening, and coming up with a subjective conclusion to help justify our are measurements?

Are some measurements more important than others? Are there trade offs to good design? How do we weigh them against each other to help reach our goals, which I would hope is better sound?

 

[Candlesticks]

The first question is whether the difference is audible and can be distinguished from another DAC. This can be done through blind ABX tests and other forms like blind preference testing.

If the difference is audible under these conditions then you'd consider whether it was worth the amount of money. A Schiit Bifrost MB costs $599 plus shipping. It's unlikely the difference is audible and if it was audible most would not consider the difference to be worth that amount of money.

The issue with slightest listening is that you don't know if what is heard is real or just placebo. Even if there is an audible difference the amount of difference may be far smaller when listened to under blind, volume-matched conditions. So blind listening is more than just about whether differences are audible but an accurate gauge of how significant the difference is. You might hear a large difference in sighted conditions and feel it was worth the money but, after blind ABX testing, come to the conclusion that the difference was far more subtle and not worth the money.

 

[sergeauckland]

As we travel down this Noble Path of measurement to better understand the quality of our audio equipment, do we actually hear what we are measuring? Or measure what we hear?

Being subjective seems to be frowned upon but, are we taking these measurements and then
(maybe unwittingly) carrying them over to listening, and coming up with a subjective conclusion to help justify our are measurements?

Are some measurements more important than others? Are there trade offs to good design? How do we weigh them against each other to help reach our goals, which I would hope is better sound?

I make measurements on my kit when I first buy it to make sure it meets spec, and/or to make sure it's good enough for use.
If an amplifier has a spec of 0.01% THD, but I measure 0.02% do I care? It's still well below audibility even if it's not to spec.
This sort of thinking I apply to everything I buy. The criterion is 'will it do the job I bought it to do? If it will, then I don't worry about it. If it won't, then either it goes back or I find a way of improving it so that it becomes adequate for the task.
In the case of ADCs and DACs, they are all so good these days, that I would be surprised if even the worse of them wouldn't be good enough, although as good ones don't cost any more than less good ones, I might as well have a good one.
S.

 

[Soniclife]

You might hear a large difference in sighted conditions and feel it was worth the money but, after blind ABX testing, come to the conclusion that the difference was far more subtle and not worth the money.

Yep, probably the fastest way to save loads of money in hifi, and be happier with your hifi.

 

[Deleted member 65]

As we travel down this Noble Path of measurement to better understand the quality of our audio equipment, do we actually hear what we are measuring? Or measure what we hear?

Being subjective seems to be frowned upon but, are we taking these measurements and then
(maybe unwittingly) carrying them over to listening, and coming up with a subjective conclusion to help justify our are measurements?

Are some measurements more important than others? Are there trade offs to good design? How do we weigh them against each other to help reach our goals, which I would hope is better sound?

There's only one more or less scientific listening test I'm aware of - The Swedish AES - Ljudtekniska sällskapet test (referred to on various posts here on ASR). It is however very time and resource consuming so I'd recommend we refrain going down the listening path.

 

[Cosmik]

Surely the questions are:

1. What is a DAC supposed to do?
2. How close can we get to a design that does what it is supposed to do, using measurements to help us?

In this case, the 'product' seems to be developed the wrong way round:

1. By what gimmick can we distinguish our 'DAC' from existing DACs?
2. Can we patch over the problems with doing it that way?
3. Can we justify how it measures by telling people that it is meant to be that way? What aggressive strategies can we employ to deter people from challenging it?

Anyone who promotes this <insert shortened name ending with 'y' desperately trying to imply instant legendary status> needs to say *why* it does what it does rather than just listening to it and saying "Glitch is inaudible". Why should people want to listen to a 'DAC' that deviates hugely from what a DAC is supposed to do?

 

[Thomas savage]

what we hear is ok, the problems lay in how we relate to it and the significance we place on perceived changes.

It’s a minefield , one Iv traveled extensively only to blow myself limb from limb till there was no sense left.

In terms of internet communication subjective impressions are non communicable. We love to express ourselves though so people enjoy painting a picture in words even if it makes no sense at all and totally fails to capture any meaningful description of what’s being heard.

All that is accomplished is a type of mutual validation pact.. fine but totally inane in terms of gaining any real insight into Audio.

That’s why we don’t care to bother with subjective listening impressions ( though a few of us might slip one in now and then , just for old times sake( said the bishop) )

 

[amirm]

One thing to note: building audio products has never been easier. So we have a ton of people getting into that business with or without competence. A lot of what my measurements (and tear downs) reveal is that competence. Given a choice, why buy a product that lacks it when another one at similar price does not?

Also, I routinely make comments about the measurement output as to whether the distortions seen are audible or not. This can only be done in spectrum graphs as that is where vast majority of our psychoacoustic knowledge exists. As noted, by that measure, many products are transparent even though they "bother the eye" to look at the data.

 

[Thomas savage]

This was posted elsewhere ..,

https://www.researchgate.net/public...n_Using_a_Multitone_Stimulus_and_Noncoherence

 

[rebbiputzmaker]

One thing to note: building audio products has never been easier. So we have a ton of people getting into that business with or without competence. A lot of what my measurements (and tear downs) reveal is that competence. Given a choice, why buy a product that lacks it when another one at similar price does not?

Also, I routinely make comments about the measurement output as to whether the distortions seen are audible or not. This can only be done in spectrum graphs as that is where vast majority of our psychoacoustic knowledge exists. As noted, by that measure, many products are transparent even though they "bother the eye" to look at the data.

Ok cool.

So on your latest test subject, you mentioned you listened for dropouts I believe, but what did you hear or not hear in relation to sound quality? Were you using revealing ancillary equipment, or just desktop stuff?

Aren't you at all curious to put both together, tests and listening, and maybe have a more holistic approach? This would be akin to say using evidence based research in order to improve healthcare outcomes.

regards

 

[amirm]

So on your latest test subject, you mentioned you listened for dropouts I believe, but what did you hear or not hear in relation to sound quality? Were you using revealing ancillary equipment, or just desktop stuff?

Listening was through my Stax headphone being driven by the Schiit BiFrost Multibit DAC. It sounded "fine." It was not a critical listening test because there was no frame of comparison and I was not looking for fidelity issues -- only drop outs.

Aren't you at all curious to put both together, tests and listening, and maybe have a more holistic approach? This would be akin to say using evidence based research in order to improve healthcare outcomes.

regards

Given infinite amount of time and resources, sure. Many people can do that work anyway. What few of them can do is measure them given the high cost and specialization in that. So that is what I am focusing my work on.

 

[rebbiputzmaker]

Listening was through my Stax headphone being driven by the Schiit BiFrost Multibit DAC. It sounded "fine." It was not a critical listening test because there was no frame of comparison and I was not looking for fidelity issues -- only drop outs.


Given infinite amount of time and resources, sure. Many people can do that work anyway. What few of them can do is measure them given the high cost and specialization in that. So that is what I am focusing my work on.

Ok. Thank you

 

[Blumlein 88]

This was posted elsewhere ..,

https://www.researchgate.net/public...n_Using_a_Multitone_Stimulus_and_Noncoherence

The AES paper about the same idea by the same authors.

http://mariobon.com/Articoli_storici_AES/Temme/2006_Non_coherent_Distortion_Temme.pdf

 

[stunta]

There are far too many DACs out there for one to realistically be able to listen and decide. Measurements help to narrow the playing field. I was in the market for a new DAC for my living room setup and I happened upon this site. The Topping D30 measured really well and the price was great, so I ordered it and liked how it sounded.

DACs like the Topping that measure well at such a low price point really help set the benchmark - a DAC costing more should measure at least as good as the Topping D30 and justify its higher price with more features - which is exactly what something like the DX7 does which was also measured by Amir.

The D30 is $120. If another manufacturer sells a DAC for the same price but doesn't measure well and Amir says there is no audible difference between the two, there are 2 reasons why I would still get the D30 -

1. If Amir can't hear a difference, that doesn't mean no one else can. If the differences are below audible levels, see #2 below
2. Why should I waste my time even trying this DAC when something at the same price measures better? I prefer to pay people who have engineered things better, even if its only on paper, purely on principle and as token of appreciation to keep up the good work.

Now if the DAC is more expensive than the D30, has the same featureset and measures worse, its even easier.

There is no art involved in music reproduction since high fidelity requires that the reproduction be as close to source as possible. Leave the art to the artists and let engineering worry about near-perfect reproduction of the art - which is music. If I hire someone to build a house for me, I want the house to be structurally sound and properly engineered. I don't want to be told something like "yeah, it doesn't quite meet building codes, but it doesn't matter - just live in it and see".

 

[March Audio]

As we travel down this Noble Path of measurement to better understand the quality of our audio equipment, do we actually hear what we are measuring? Or measure what we hear?

Being subjective seems to be frowned upon but, are we taking these measurements and then
(maybe unwittingly) carrying them over to listening, and coming up with a subjective conclusion to help justify our are measurements?

Are some measurements more important than others? Are there trade offs to good design? How do we weigh them against each other to help reach our goals, which I would hope is better sound?

Being subjective is not frowned upon. However subjective conclusions as drawn by most home users, or as seen on most fora or magazines where the DUT is sighted and where the subjective tests are uncontrolled is. My experience is that the moment you put audiophiles under the most unobtrusive of controls their ability to discern these alleged massive differences between components evaporates.

This bias effect, as you allude to can of course operate the other way. Schitt products really do appear to measure shit. Does that mean the measurer will now perceive the sound quality to be shit? Its quite possible without controls. However most serious contributors to this forum are well aware of the issues of bias and would not embark on a serious listening test without controls in place - even as simple as making sure that the device playing was not known helps. Thats the difference of what we do on this forum.

I have just measured a R2R dac elsewhere in another thread. It has the typical technical issues of the R2R methodology, ie high harmonic distortion. I still think it sounds OK - quite good in fact. Are the issues below audibility? Possibly, but I would have to do some serious controlled listening tests to be sure.

I also have sat on my desk a $40 dac board which measures quite well. Its no surprise to me that it also sounds good. So why would I spend $600 on a Schitt DAC that has appalling measurements? It doesnt make sense.

Then of course we come across the issue where some people actually subjectively like the sound of certain technically poor equipment. Turntables are the prime example. Technically a pile of poo on any level however some like the sound. No criticism of that, but it shows how difficult it is to put any persons subjective personal opinions into context, let alone any relation to accurate reproduction (hifi).

 

[tomelex]

There is a very gray area between measurements and subjective opinions in my book. Measurements simply reveal fidelity to the signal. However, and this is key, fidelity to signal when you have a system of components does not equate directly to what I feel sounds best. Sounding best is a personal thing.

Depending on the mood I catch you in, I can play a SET amp and a solid state amp, and one time you would prefer the SET amp and the next the solid state amp. It just might be that the SET amp in a system has second harmonic levels that when matched to your speakers second harmonic levels end up cancelling each other and you might perceive that the cleaner resulting sound at your ears means better fidelity, or not.

Yet, if you do like a bit of a syrupy sound, we could measure the harmonic spray of some gear and find gear with a similar harmonic spray and in that way measurements would indeed correlate to what you like to hear.

 

[Wombat]

There is a very gray area between measurements and subjective opinions in my book. Measurements simply reveal fidelity to the signal. However, and this is key, fidelity to signal when you have a system of components does not equate directly to what I feel sounds best. Sounding best is a personal thing.

Depending on the mood I catch you in, I can play a SET amp and a solid state amp, and one time you would prefer the SET amp and the next the solid state amp. It just might be that the SET amp in a system has second harmonic levels that when matched to your speakers second harmonic levels end up cancelling each other and you might perceive that the cleaner resulting sound at your ears means better fidelity, or not.

Yet, if you do like a bit of a syrupy sound, we could measure the harmonic spray of some gear and find gear with a similar harmonic spray and in that way measurements would indeed correlate to what you like to hear.

I basically agree with you. I have no problem with individuals having tonal preferences. I cannot agree with those who claim equipment which inherently adds audible coloration to the signal is HiFi.

 

[Dismayed]

I find it funny that subjectivists will almost never submit to blind tests. Sorry, but if you can't differentiate equipment with your eyes shut, then you have no credibility.

 

[Blumlein 88]

I think the title of the thread falls into what I have seen called a mu answer. The question is too simple and the answer too big. Or question is too short so no answer is it. It isn't either or but something more. Or even such question should only be answered with "unask the question" because the question is in error as asked.

We do not hear what we measure. We also do not measure what we hear. To conclude there is no sensible answer however is incorrect. The question needs work before the answer can have value.

 

[rebbiputzmaker]

I think the title of the thread falls into what I have seen called a mu answer. The question is too simple and the answer too big. Or question is too short so no answer is it. It isn't either or but something more. Or even such question should only be answered with "unask the question" because the question is in error as asked.

We do not hear what we measure. We also do not measure what we hear. To conclude there is no sensible answer however is incorrect. The question needs work before the answer can have value.

The title is not the entire question, the text supports it. It is compound and complex and can be multi-threaded. It can require full honest reflection, and a holistic approach. Most things do not exist in a vacuum imo. i.e. Your name alone does not define who you are.

https://surnames.behindthename.com/names/source/occupation