The relevance of measurements to audible quality of sound

[Magnum Innominandum]

Hello,

In this forum we see specific measurements posted regarding many devices.

Recommendations are made on the basis of these measurements. All very scientific, which I like.

I mean who would buy a Car on a vague concept "how it drives"?

We know top speed, torque, mpg etc. and can rely on these together with convenience features (how many cup holders, size of the trunk and how easy to access) a shape and colour that pleases us to get a good choice.

No need to test drive, right?

Now with a car I can follow the value of these measurement numbers. I can understand why a car with good torque, top speed, mpg and convenient layout would be reasonably recommended over one that falls down in one or several of these areas, though it has not stopped the buying public from buying inconvenient and inefficient cars, myself included and yes, I test drive and decide ultimately on many factors including how it feels driving on what I buy.

While the measurements presented here seem of a similar kind (Distortion, Noise, Jitter), I would ask for the scientific background.

If we elevate specific qualities as a measure of the recommendability of a device to listen to music with, there ought to be good scientific evidence that shows that lower distortion is always better sounding, that lower noise is always better sounding and I mean not in a relative sense as in: "Above a certain limit lower distortion always sound better" or "Above a certain limit lower noise always sounds better", but in the absolute sense in which it is presented here.

Alternatively, if these is a consensus about what levels of distortion, noise and jitter are actually audible, it would be interesting to know what they are and how tested devices results compare to those limits.

Also, if there are such limits and all tested devices are below the audibility threshold, should recommendations based on the measurements still be made?

Ideally the proponents of "low distortion uber alles", "low noise uber alles" and "low jitter uber alles" simply have reliably scientific tests to present that support their position and then we can continue measuring and recommending on this basis. My own research failed to provide such tests and evidence, however surely no-one scientific inclined would test result based recommendations without good and solid evidence that these test results are meaningful in the context.

Magnum Innominandum

 

[Krunok]

Excellen post, it even has car analogy..

There's only one problem here - that question have already been asked but no answers were given. It seems that some vague consensus about audible threshold limits for various measurement parameters might exist among "elders" on this forum based on their professional experience with audio, but it has not been defined.

 

[Wombat]

Are you aware of the test and analysis technology in motor racing. The degree of car performance predictability is extremely high. They do test drives for comparison and the driver gives his objective and subjective feedback.

Except for the intensity and complexity of racing car design and use, the practises parallel audio design and performance. Some drivers admit to superstitious beliefs - a similarity with 'audiophiles'?

 

[Wombat]

Excellen post, it even has car analogy..

There's only one problem here - that question have already been asked but no answers were given. It seems that some vague consensus about audible threshold limits for various measurement parameters might exist among "elders" on this forum based on their professional experience with audio, but it has not been defined.

Audible threshold limits are no mystery from a testing point of view and they are documented. It is just that some people refuse to accept them without putting up convincing challenges.

 

[Krunok]

Audible threshold limits are no mystery from a testing point of view and they are documented. It is just that some people refuse to accept them without putting up convincing challenges.

Are they really no mistery? Because, for example, I remember we had an "issue" with unbalanced outputs of poor SMSL VMV V1 DAC and I don't remeber anybody clearly stated if that "issue" is audible or not.

Anyway, if audible threshold limits are not mistery please state them here.

 

[Wombat]

Are they really no mistery? Because, for example, I remember we had an "issue" with unbalanced outputs on poor SMSL VMV V1 DAC and I don't remeber anybody clearly stated if that "issue" is audible or not.

So?

Are we getting into Plankton territory again.

 

[Krunok]

So?

Are we getting into Plankton territory again.

As I said, if audible threshold limits are not mistery please state them here.

 

[Magnum Innominandum]

Audible threshold limits are no mystery from a testing point of view and they are documented.

That is excellent. Do you mind pointing me to the documentation?

Excellen post, it even has car analogy..

There's only one problem here - that question have already been asked but no answers were given. It seems that some vague consensus about audible threshold limits for various measurement parameters might exist among "elders" on this forum based on their professional experience with audio, but it has not been defined.

I am hoping the measurement meister himself, Amir would grace us with his take on what is audible and why and how audible for example the SINAD or Noise level differences he measures are in reality and how much we should worry about them and/or base our purchasing decisions upon them.

Naturally if SINAD had a direct proven correlation with "good sound" we should buy the most SINAD our budget allows, right? But if there is a limit below which improved SINAD no longers improves sound quality that should be stated. We may be able to lower our budget by just buying enough SINAD to not affect sound quality.

Worse would be, what if there is no proven correlation between SINAD and Sound Quality? I have tried my best to find any scientifically solid paper or material that reliably links SINAD with "Good Sound" but came up empty. Surely someone scientifically inclined like Amir would not make recommendations based on SINAD if there was no proof that such a link exists.

The same naturally applies to the other Measures? We can measure, but where is proof that what is measured actually matters? And within what limits if any?

Magnum Innominandum

 

[solderdude]

Hello,

While the measurements presented here seem of a similar kind (Distortion, Noise, Jitter), I would ask for the scientific background.

If we elevate specific qualities as a measure of the recommendability of a device to listen to music with, there ought to be good scientific evidence that shows that lower distortion is always better sounding, that lower noise is always better sounding and I mean not in a relative sense as in: "Above a certain limit lower distortion always sound better" or "Above a certain limit lower noise always sounds better", but in the absolute sense in which it is presented here.

Alternatively, if these is a consensus about what levels of distortion, noise and jitter are actually audible, it would be interesting to know what they are and how tested devices results compare to those limits.

Also, if there are such limits and all tested devices are below the audibility threshold, should recommendations based on the measurements still be made?

Ideally the proponents of "low distortion uber alles", "low noise uber alles" and "low jitter uber alles" simply have reliably scientific tests to present that support their position and then we can continue measuring and recommending on this basis. My own research failed to provide such tests and evidence, however surely no-one scientific inclined would test result based recommendations without good and solid evidence that these test results are meaningful in the context.

Magnum Innominandum

Well in general the better the performance the closer the devices are to the 'truth'. Truth here being the output of a device does not alter the signal (other than intended).

There is no such thing as a jitter number that says something about audibility of such. Reason the spectrum of it (lower or higher frequency jitter) nor whether or not the jitter is related to the signal or random and the ability of the listener to discriminate which is not the same as thinking/being convinced one hears it.

There is no such thing as a distortion number that tells everything about its audibility. This depends on the frequency, spectrum, type of distortion and amplitude as well as the ability of the listener to detect it and be bothered by it, which is not the same as thinking/being convinced one hears it.

Such is true for all numbers in audio. One has to know what they mean and how the numbers are created. I know for a fact that many manufacturers just put meaningless numbers in their spec sheets.
Do you want to go by manufacturer specs or measured specs. Do you trust measurements of some random twat on the internet or someone you trust ?

Is the device tested under realistic conditions or not is another question. Are the proper limits applied for the tests ? (Freq. response can be given at -0.1db, +/- 3dB, +/- 10dB ... 20 dB ?)

In general though... the better the spec the more likely the device does not change the sound.
But what if you are after certain coloration/changes in the sound (often in a euphonic way) which specs should be looked at.

Specs are merely a guide that may tell someone (who knows what they mean) something about the (usually maximum) limits and performance.
It is often 'misused' to slap measurebators by telling them how wrong they are using specs while ears are much more discriminative.
Ears aren't but at the same time measurements can be wrong as well.
One or 2 numbers don't fully characterize a device. A total suite (set) of measurements can say something to those that undestand the suite.
When one doesn't understand ... well don't bitch or complain about it and either learn or move on.

Proper measurements are useful but not to everyone !

Want to know what matters and what your borders are... investigate your own hearing abilities ... but do it properly. That last bit is where things get tricky

 

[Wombat]

As I said, if audible threshold limits are not mistery please state them here.

Do some homework. Google is a good starting place.

 

[Wombat]

As I said, if audible threshold limits are not mistery please state them here.

As I said in other words, find out for yourself. With your attitude don't expect me to feed an armchair enthusiast.

 

[andreasmaaan]

@Magnum Innominandum this is one of my favourite topics, really fascinating stuff.

@Krunok is right to some extent, there are no completely clear answers here, but I also tend to agree with @Wombat: these things are not a great mystery; there's just some margin for error in those thresholds that have been experimentally derived.

There are lots of threads on this in the Psychoacoustics section, which is one place I'd suggest you start.

If you haven't already, I'd also strongly recommend reading Floyd Toole (either his whole book or his various online articles) as an introduction, then this classic text for a more in-depth understanding.

FWIW, I'm going to do something probably controversial (and certainly overly simplistic) and answer Krunok's question by listing here what I consider to be the most conservative estimates of audibility thresholds based on solid experimental data. Most of these thresholds will be higher in practice/reality, but these are worst-case-scenario experimentally obtained thresholds (to the best of my knowledge):

• Non-linear distortion: 0.003% when distortion is of the most heinous kind, higher in most typical cases, very SPL dependent (achievable with electronics, impossible with speakers)
• Group delay: 0.8ms above 300Hz with speakers; lower with headphones, higher at lower frequencies (achievable)
• Jitter: 10ns for sine waves (frequency dependent), 30ns or more for music (achievable)
• Noise: -149dB at 2-3KHz (but only if you're listening in an anechoic chamber at a level so loud it literally causes pain and permanently damages your hearing), lower at other frequencies, much lower in all practical cases (achievable)

Sources (apart from those mentioned above):

• http://www.ultrahighendreview.com/uploads/documents/Benjamin_and_Gannon_on_Jitter.pdf
• http://vbn.aau.dk/en/publications/o...ns(1427f370-cc89-11db-9e19-000ea68e967b).html
• https://www.researchgate.net/public...SHOLD_OF_TRANSIENT_INTERMODULATION_DISTORTION
• http://www.aes.org/e-lib/browse.cfm?elib=3824
• https://secure.aes.org/forum/pubs/conventions/?elib=19404
• http://digitool.library.mcgill.ca/R/?func=dbin-jump-full&object_id=141279&local_base=GEN01-MCG02
• ...and many more. AES is a good place to start but membership is expensive...

 

[Magnum Innominandum]

Well in general the better the performance the closer the devices are to the 'truth'. Truth here being the output of a device does not alter the signal (other than intended).

I would call this a truism. Naturally, lower dissolved solids in water make purer water, but for drinking water there are recommendation on not only how much is too much but also how little is "too pure".

My question was if there is any scientific proof that "the lower a given measurement of deviation from ideal the better the sound", which I feel would be needed to make recommendations on the basis of such a measurement.

There is no such thing as a jitter number that says something about audibility of such. Reason the spectrum of it (lower or higher frequency jitter) nor whether or not the jitter is related to the signal or random and the ability of the listener to discriminate which is not the same as thinking/being convinced one hears it.

There is no such thing as a distortion number that tells everything about its audibility. This depends on the frequency, spectrum, type of distortion and amplitude as well as the ability of the listener to detect it and be bothered by it, which is not the same as thinking/being convinced one hears it.

Such is true for all numbers in audio. One has to know what they mean and how the numbers are created. I know for a fact that many manufacturers just put meaningless numbers in their spec sheets.

.........................

One or 2 numbers don't fully characterize a device. A total suite (set) of measurements can say something to those that undestand the suite.
When one doesn't understand ... well don't bitch or complain about it and either learn or move on.

So you are saying the measurements presented here and the recommendations made based upon them are not "a thing".

As consumer I would like some reliable guide in selecting my DAC, headphone amp and headphone. Clearly random subjective reviews that do not even level match and use a minimum of reducing expectation bias cannot be trusted.

Now I read from your words that if I select the highest SINAD, lowest jitter and lowest noise DAC/Amp my budget can accommodate, this does not guarantee me the best sound for my money? Then what value do these measurements have? What do they tell me?

This site presents itself as offering honest and scientific based recommendations. I should not need to learn more, or have to have a massive suite of difficult to interpret measurements for that.

I should be shown the measurements, it should be demonstrated how and why they matter and I should be able to buy what measures best and get what sounds best. Otherwise this site and it's measurements have no greater value than reading the reviews at HeadFi.

Do some homework. Google is a good starting place.

I have googled and could not find anything. As you stated so categorically audibility limits are know and documented I was hoping for a link to the documentation so I can check what they are and hopefully assess how reliable these limits may be from the documentation. Again, this site explicitly promotes measurements as basis of purchasing recommendations - so surely I am not the one who has to find out if these measurements are meaningful or relevant?

Magnum Innominandum

 

[Magnum Innominandum]

• Non-linear distortion: 0.003% when distortion is of the most heinous kind, higher in most typical cases (achievable with electronics, impossible with speakers)
• Group delay: 0.8ms above 300Hz with speakers; higher at lower frequencies (achievable)
• Jitter: 10ns for sine waves (frequency dependent), 30ns or more for music (achievable)
• Noise: -149dB at 2-3KHz (but only if you're listening in an anechoic chamber at a level so loud it literally causes pain and permanently damages your hearing), lower at other frequencies, much lower in all practical cases (achievable)

So if I read you read you right, a SINAD of 90dB or better for the "most heinous kind of distortion" is inaudible? So better than 90dB SINAD does not matter?

Group delay I read about here, it is not even measured here. Do I need to worry about this?

Jitter levels measured here tend to be way, way lower than 10nS so in effect almost all devices here have low enough jitter to be sonically transparent?

Noise, I talked about this once with a recording engineer. I talked about high levels of SNR and he laughed at me and gently explained that most recording microphones have around 26dB self noise and a maximum SPL of 130dB making the dynamic range that can be recorded via most recording microphones less than 110dB or 18 Bit. He then added that in reality even less than the 104dB in our theory can be reached because often recording microphones are placed at distances from instruments where the maximum SPL is not reached. He also pointed out that mixing multiple microphones together further increases the noise.

He then asked me if I knew the background noise in my living room. I said I did not, h suggested in a superbly quiet room at night I might have 20dB background noise, usually more. If listening at 105dB peak SPL (THX standard) this gives only 85dB dynamic range, which is less than CD offers and much less than most devices tested here.

If I was using noise cancelling headphones I might get lower noise, but I feel the sound from the Bose QC-15 I keep for traveling is very unnatural and I feel they have some noise I can hear when the room is quiet. I like open back headphones (I bought the Massdrop Sennheisers) much better. But they do not reduce noise at all.

So noise levels of better than perhaps also -90dB to -100dB would seem "good enough" in practice, given the limitations of recording and room noise.

In this case it would seem almost all the devices here measure "good enough" and cannot be recommended over each other based on measurements!?

If so, should this not be stated somewhere to avoid misleading people into buying a higher spec than they need?

In another post it was intimated that there are other factors that relate to good sound which are not covered by measurements here (you added group delay), should they not be added and limits where known be stated?

All in all I get a feeling that there is less hard science here at work, but instead a belief into the magical power of measurements and I am left as bewildered and confused as when reading the subjective reviews all over.

It makes deciding on what DAC/Amp I need hard. Maybe the Headphone Jack on my Laptop is good enough? But it does not play loud enough with the Sennheisers. How can I choose the best DAC/Amp?

Magnum Innominandum

 

[solderdude]

My question was if there is any scientific proof that "the lower a given measurement of deviation from ideal the better the sound", which I feel would be needed to make recommendations on the basis of such a measurement.

Ain't got nothing to do with taste. When an input signal is amplified or reproduced and the SHAPE of the waveform is altered the least (except for the properties that need to change) then the 'fidelity' is highest... regardless of what one likes or prefers.

So you are saying the measurements presented here and the recommendations made based upon them are not "a thing".

No that's what you are saying. Misinterpretation of measurements also is quite common. recommendations are personal or based on 'less changes = better'

As consumer I would like some reliable guide in selecting my DAC, headphone amp and headphone. Clearly random subjective reviews that do not even level match and use a minimum of reducing expectation bias cannot be trusted.

That may depend on WHO does the subjective review and how that evaluation is done and ain't got nothing to do with measurements so don't see the relevance with measurements.

Now I read from your words that if I select the highest SINAD, lowest jitter and lowest noise DAC/Amp my budget can accommodate, this does not guarantee me the best sound for my money? Then what value do these measurements have? What do they tell me?

When measurements don't tell you anything you need/want to know just disregard them.
Nobody is forcing anybody to understand the measurements.
You either want or can understand the whole measurement suite or you don't.
Learn it or forget it.
Don't get hung up on a single number from any measurement. Not Sinad, not THD, not FR ...

This site presents itself as offering honest and scientific based recommendations. I should not need to learn more, or have to have a massive suite of difficult to interpret measurements for that.

No you don't unless you WANT to understand what measurements mean. Take the recommendations for what they are. Also take the recommendations of Gurus for what they are. Take my recommendations for what they are and I will do the same with yours and those of others.

I should be shown the measurements, it should be demonstrated how and why they matter and I should be able to buy what measures best and get what sounds best. Otherwise this site and it's measurements have no greater value than reading the reviews at HeadFi.

Or you could be shown the measurements and understand what matters (to you).
Or you could look at the recommendations and dismiss them or not, trust someone else's recommendations instead or make up your own mind.
Indeed this site, too many folks, will have no greater value than head-fi or subjective based sites.
So what ?

 

[Wombat]

I would call this a truism. Naturally, lower dissolved solids in water make purer water, but for drinking water there are recommendation on not only how much is too much but also how little is "too pure".

My question was if there is any scientific proof that "the lower a given measurement of deviation from ideal the better the sound", which I feel would be needed to make recommendations on the basis of such a measurement.



So you are saying the measurements presented here and the recommendations made based upon them are not "a thing".

As consumer I would like some reliable guide in selecting my DAC, headphone amp and headphone. Clearly random subjective reviews that do not even level match and use a minimum of reducing expectation bias cannot be trusted.

Now I read from your words that if I select the highest SINAD, lowest jitter and lowest noise DAC/Amp my budget can accommodate, this does not guarantee me the best sound for my money? Then what value do these measurements have? What do they tell me?

This site presents itself as offering honest and scientific based recommendations. I should not need to learn more, or have to have a massive suite of difficult to interpret measurements for that.

I should be shown the measurements, it should be demonstrated how and why they matter and I should be able to buy what measures best and get what sounds best. Otherwise this site and it's measurements have no greater value than reading the reviews at HeadFi.



I have googled and could not find anything. As you stated so categorically audibility limits are know and documented I was hoping for a link to the documentation so I can check what they are and hopefully assess how reliable these limits may be from the documentation. Again, this site explicitly promotes measurements as basis of purchasing recommendations - so surely I am not the one who has to find out if these measurements are meaningful or relevant?

Magnum Innominandum

Re your learning, try starting at the bottom and work up. You are involved beyond your ken. This is not unusual in this world of instant-knowledge expectation and an expectation for simple answers. Knowledge gained from audio forums is usually fragmented and reduced to excess simplicity which cannot be related to the broader understanding of the science and practise. There is only so much that can be done to answer naive questions. Knowledgable members, here, try but their input is often rejected or further disputed.

You happened to post when I am particularly pissed at amateurs disparaging the experience, knowledge and expertise of those who put in years of learning and dedicated practise in their technical professions.

Those arrogant demands and objections by the technically deficient are an insult. I will not be dragged down to that level.

End of rant.

 

[andreasmaaan]

Group delay I read about here, it is not even measured here. Do I need to worry about this?

Electronics don't typically introduce significant group delay, so this is a problem to think about with speakers only.

In this case it would seem almost all the devices here measure "good enough" and cannot be recommended over each other based on measurements!?

The devices here are measured at 0dBfs or just below, which generally gives a best case scenario for noise and nonlinear distortion, and at specific frequencies only. The measurements give an excellent indication of how a device will perform under a variety of conditions, but are not sufficiently rigorous to know how a device will perform in all use cases. And distortion and noise rise at lower levels (as shown in the IMD vs dB measurement).

But yes, many DACs measured here seem to be good enough to not exceed the experimentally derived thresholds in most use cases.

 

[Magnum Innominandum]

Re your learning, try starting at the bottom and work up. You are involved beyond your ken.

All I want and need is reliable information to help me make the right purchasing decision for a DAC/Amp for my Massdrop Sennheiser Headphones.

It seems that other having enough output to drive them there is nothing to be gleaned from the measurements here that helps. That is quite dispiriting.

Magnum Innominandum

 

[Wombat]

All I want and need is reliable information to help me make the right purchasing decision for a DAC/Amp for my Massdrop Sennheiser Headphones.

It seems that other having enough output to drive them there is nothing to be gleaned from the measurements here that helps. That is quite dispiriting.

Magnum Innominandum

Test threads on this forum have covered this ad nauseam. Why didn't you simply state this up-front?

Over to a headphone guy.

 

[Blumlein 88]

You firstly must pick apart better or best sound and accuracy.

Accuracy we can investigate quite thoroughly. How accurate is audibly accurate is less clear, but there are some good signposts.

Better is another issue altogether. As some have preferences which may differ from accuracy. You may prefer the red car. The red car will have the same performance as an otherwise identical blue car. You still might prefer red. Some cars may have an inferior performance which engages the driver in a way they prefer. Think a manual transmission vs a faster automatic transmission. Audio gear may have a color that engages the listener in some way so they prefer it to more accurate gear.