• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Double Blind tests *did* show amplifiers to sound different

TREVANIAN

Member
Joined
Aug 24, 2020
Messages
5
Likes
6
Total noob here, but if I assume all amps sound the same given same level etc, then what drives the amp designers to continue. Do they know, or have they convinced themselves that amps sound different. I guess one can also assume some amp designers are money hungry and don't care, but what about the genuine ones that are striving to make better amps, why would they if they did sound the same? I have zero background and very little experience with expensive hifi amps but I don't see why they would sound the same. Just wondering thats all.
 

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,598
Likes
25,500
Location
Alfred, NY
Total noob here, but if I assume all amps sound the same given same level etc, then what drives the amp designers to continue. Do they know, or have they convinced themselves that amps sound different. I guess one can also assume some amp designers are money hungry and don't care, but what about the genuine ones that are striving to make better amps, why would they if they did sound the same? I have zero background and very little experience with expensive hifi amps but I don't see why they would sound the same. Just wondering thats all.
Size, efficiency, price point, features, and of course, power. In some cases, the challenge of taking distortion and noise down to the atomic level, which of course is irrelevant to audibility.

Just because a problem is solved does not remove commercial incentives.
 

MakeMineVinyl

Major Contributor
Joined
Jun 5, 2020
Messages
3,558
Likes
5,878
Location
Santa Fe, NM
Hmmm, so are we worshiping you, or cursing you?!?:D
I think you posted this in the wrong thread....

I certainly don't want to be worshiped by any entity other than my cat. He probably curses me too......
 

DonH56

Master Contributor
Technical Expert
Forum Donor
Joined
Mar 15, 2016
Messages
7,945
Likes
16,791
Location
Monument, CO
I think you posted this in the wrong thread....

I certainly don't want to be worshiped by any entity other than my cat. He probably curses me too......

You are sooo screwed...
 

Mulder

Addicted to Fun and Learning
Forum Donor
Joined
Sep 2, 2020
Messages
646
Likes
896
Location
Gothenburg, Sweden
I'm not anti-science. Good scientists know that all science is subject to revision.

Yes, but only under certain methodological conditions. A scientific conclusion, which falsifies a previous conclusion, must live up to objective criteria and be possible to repeat, ie other researchers must be able to reach the exact same conclusion. This is precisely to eliminate subjective mistakes or other bias.

This whole discussion leads to the question: Is audio technology something that can be understood and designed on the basis of scientific criteria and methods, or is it a kind of art form. I believe that those who claim that scientific explanations are not enough have the burden of proof to show exactly WHAT it is that is insufficient with the measurements made today. Exactly what are these not able to measure?

Now that I have read through this thread, it is my impression that your argumentation leads to the following conclusion:
  • Cognitive bias does not exist when it comes to the assessment of sound reproduction, or if it exists, the risk that it leads to erroneous conclusions is so low that it can be ignored. There is no risk that perceived differences are objectively incorrect.
If it is correct, ie that I have understood you correctly, then it must be pointed out that it is a conclusion that is completely contrary to what we know about how people perceive the world through their senses.
 
Last edited:

Feanor

Senior Member
Forum Donor
Joined
Feb 22, 2019
Messages
382
Likes
497
Location
southwestern Ontario
Yes, but only under certain methodological conditions. A scientific conclusion, which falsifies a previous conclusion, must live up to objective criteria and be possible to repeat, ie other researchers must be able to reach the exact same conclusion. This is precisely to eliminate subjective mistakes or other bias.

This whole discussion leads to the question: Is audio technology something that can be understood and designed on the basis of scientific criteria and methods, or is it a kind of art form. I believe that those who claim that scientific explanations are not enough have the burden of proof to show exactly WHAT it is that is insufficient with the measurements made today. Exactly what are these not able to measure?

Now that I have read through this thread, it is my impression that your argumentation leads to the following conclusion:
  • Cognitive bias does not exist when it comes to the assessment of sound reproduction, or if it exists, the risk that it leads to erroneous conclusions is so low that it can be ignored. There is no risk that perceived differences are objectively incorrect.
If it is correct, ie that I have understood you correctly, then it must be pointed out that it is a conclusion that is completely contrary to what we know about how people perceive the world through their senses.
So I'm not a scientist just a poor dope who is trying to understand why I hear differences between, say, amplifiers.

The standard answer, I'm sure you've heard, is that it's my expectations, (cognitive bias), which, as best, causes confirmation bias or, worse, causes me tp just imagine things. I often do have expectation of what a device may sound like, accordingly it would be easy for me to believe that was the whole explanation if it weren't for fact that devices sound significantly different to me than what I expected.

Good and proper science demands that experiments be relevant, objective, and repeatable. Experiments are design with these objectives in mind. A common confounding factor is that adequately meeting the latter criteria, (objectivity & repeatability), reduces the real-world relevance of the experiment. I'm assured that it is well proven in objective, repeatable bling ABX experiments that test subjects cannot reliably distinguish one amplifier from another. (which of course is statistical: 97% of the time, +/- whatever percent)

Know what? I believe this result. The problem is that audiophiles who insist they hear differences do that under conditions don't remotely conform to the rigor blind ABX design.
 

Sal1950

Grand Contributor
The Chicago Crusher
Forum Donor
Joined
Mar 1, 2016
Messages
14,309
Likes
17,145
Location
Central Fl
Know what? I believe this result. The problem is that audiophiles who insist they hear differences do that under conditions don't remotely conform to the rigor blind ABX design.
And as such are rendered meaningless in any serious investigation of the claims.
 

Feanor

Senior Member
Forum Donor
Joined
Feb 22, 2019
Messages
382
Likes
497
Location
southwestern Ontario
And as such are rendered meaningless in any serious investigation of the claims.
No dispute: their claims are not scientific. What would be sensible would be for the serious scientists to devise tests for conditions that are a lot more similar to actual audiophile listening methods.
 

Killingbeans

Major Contributor
Joined
Oct 23, 2018
Messages
4,103
Likes
7,613
Location
Bjerringbro, Denmark.
A common confounding factor is that adequately meeting the latter criteria, (objectivity & repeatability), reduces the real-world relevance of the experiment.

The problem is that audiophiles who insist they hear differences do that under conditions don't remotely conform to the rigor blind ABX design.

What would be sensible would be for the serious scientists to devise tests for conditions that are a lot more similar to actual audiophile listening methods.

Not very scientific, but how about this?: Buy two of the same model of a highly praised boutique amp. Find a cheap PA amp with similar gain, power output and clipping behavior (EDIT: and same level of residual noise). Gut one of the boutique amps and replace the innards with those from the PA amp. Modify the Franken-amp to have same frequency responce and in/output impedance as the virgin boutique amp. Give the two seemingly identical amps to a handful of Audiophiles and ask them to identify the "heavily modified" one using whatever listening method they want.

I bet it would be completely impossible for them to do so.
 
Last edited:

MakeMineVinyl

Major Contributor
Joined
Jun 5, 2020
Messages
3,558
Likes
5,878
Location
Santa Fe, NM
Not very scientific, but how about this?: Buy two of the same model of a highly praised boutique amp. Find a cheap PA amp with similar gain, power output and clipping behavior. Gut one of the boutique amps and replace the innards with those from the PA amp. Modify the Franken-amp to have same frequency responce and in/output impedance as the virgin boutique amp. Give the two seemingly identical amps to a handful of Audiophiles and ask them to identify the "heavily modified" one using whatever listening method they want.

I bet it would be completely impossible for them to do so.
I'll get on that right away.
 

solderdude

Grand Contributor
Joined
Jul 21, 2018
Messages
16,158
Likes
36,894
Location
The Neitherlands
Not very scientific, but how about this?: Buy two of the same model of a highly praised boutique amp. Find a cheap PA amp with similar gain, power output and clipping behavior (EDIT: and same level of residual noise). Gut one of the boutique amps and replace the innards with those from the PA amp. Modify the Franken-amp to have same frequency responce and in/output impedance as the virgin boutique amp. Give the two seemingly identical amps to a handful of Audiophiles and ask them to identify the "heavily modified" one using whatever listening method they want.

I bet it would be completely impossible for them to do so.

Did something similar about 30 years ago already. Fun and educational. I didn't even bother to change output resistance and FR.
The biggest giveaway would be the difference in warm-up time between these amps. I 'solved' this with an output mute relay with a 30 second delay.
 

Mulder

Addicted to Fun and Learning
Forum Donor
Joined
Sep 2, 2020
Messages
646
Likes
896
Location
Gothenburg, Sweden
So I'm not a scientist just a poor dope who is trying to understand why I hear differences between, say, amplifiers.

Well you hear it because your brain think it is what it hear. Why? Maybe what you hear is a real difference. Or you hear a difference because you expect to hear a difference. Who am I to know? We hear with our brains, and the brain never register only an isolated sensory impression, as in a measuring device. In the brain, hearing impressions are constantly mixed with other impressions and with our stored experiences and prejudices. On the personal level, it is very difficult to sort out what we hear from other impressions. Some differences are of course obvious. But in HiFi we mostly talk about small subtle differences.

I believe that you confuse personal experiences and impressions with what constitutes a scientific method. It may be difficult and not practical for the individual consumer to carry out scientifically based evaluations, but one could wish that HiFi magazines and reviewers would avoid the purely subjective and rely on scientific methods.

What I find strange in your argument is that you claim that audiophiles hear differences when listening is not performed under controlled forms, because the controlled forms affect what we hear. You then conclude that it is the scientifically controlled situation that leads to the wrong conclusion, not that it is in the uncontrolled situation that mistakes or bias arise. You therefore turn the burden of proof, and dismiss the objective with the subjective. That's a problematic argument. How do you know it's not exactly the opposite?

The problem we deal with, when it comes to scientific methods and determine what is good hi-fi equipment, is to bridge the risk of subjective impressions. Consequently, we perform controlled listening tests, or we measure the properties of the devices. If you have objections to this, then you must specify on a principal level what it is that cannot be measured, and why it on a principal level would be safer with subjective impressions than with controlled listening tests. After all, audio is a physical and not a magical reality, and there are no oddities involved in this. However, there are many parameters to take into account, and the discussion of the balance between these parameters is of course relevant, but it must then be based on real and objectively established parameters and not imagined subjective opinions.

I myself have participated in and carried out controlled listening tests, and I am often amazed how the listeners can deceive themselves when they think they are listening to something other than what they are actually listening to. There are lots of tests that show the same thing. For example, many people prefer the sound of vinyl, when they think it is vinyl they are listening to, but when the sound is in fact digital. Or. Many people hear a sound improvement when they think they are listening to a product that pretends to provide cleaner electricity, such as an exclusive power cable, even though it is in fact a simple standard cable they are listening to.

It is perfectly okay to have subjective perceptions and personal favors. No one can demand of anyone else that they report their experience in scientific terms. It's no problem. The problem arises when the industry, not infrequently assisted by HiFi magazines and reviewers, manipulates consumers. I have been to hifi fairs or other so-called test, where representatives of the products clearly manipulated the audience. This may be conscious, or unconscious, but reflects the total absence of a critically grounded scientific approach. We know, for example, that network switches cannot affect the sound in a DAC. Still, I have participated in demonstrations where the audience still hears differences. Why? Because they expect differences. Psychological cognitive bias in other words. And often, even those who are aware of these risks are not safe from this bias. I myself have on several occasions found myself hearing differences where such have not existed. What the heck! I think my HiFi sounds better in the evening. And if I turn off the light, it sounds even better.
 
Last edited:

jomark911

Active Member
Joined
Feb 14, 2020
Messages
125
Likes
55
Location
Greece
I have been invited recently , by a well known person , here in Athens Greece, to audition , my own same amp on his own speakers.
Controversy if I say it right is that he believes that the same amp sounds much worse than mine.
The catch here is that my system is a multi amped active system ,including a electronic crossover network and three amps , one for each way.
While his system is normal passive speakers system , driven by the same amp.
He's diaphony (greek word meaning disagreement ) was that my system sounded overly super analytical , while his was a more laid back , easy to listen to .
While I explained to him , there is nothing equal to compare to , he kept on .
We still are friends but from a distance.
You can not compare apples to oranges my good friend. There are some huge differences in our systems even if we own the same amp.
I use this amp just for bass reproduction , whilst you are using it for the entire spectrre.
My amp is working like high principles while yours is working like overtimes .
My system is based on a totally different architecture compared to yours.
Yes we do have the same amp , this is the only common thing that we have.
From there on there is nothing else that is the same. Different space , different qonfig different space , different speakers , different everything.
Having a pair of speakers like my friends , up in the attic , I must confess that auditioning them with the same amp brought up some memories but not exactly the same sound . Mine was a bit more harsh sterile I would say.
Naturally it was NOT the amp performing differently , but the room along with the speakers.
I tried to talk him this . No answer. He still insists that my amp performs better.
Well there are some things that get no remedy , no rectification. That's life , and it sucks .
 

Ingres3225

New Member
Joined
Jul 27, 2020
Messages
4
Likes
4
Size, efficiency, price point, features, and of course, power. In some cases, the challenge of taking distortion and noise down to the atomic level, which of course is irrelevant to audibility.

Just because a problem is solved does not remove commercial incentives.

So, if I, your average guy with no technical know-how, who has a Cambridge Audio CXA60 driving a pair of Klipsch Forte IIIs, wants to choose a new integrated amplifier, am I decieving myself into thinking a Sugden IA-4 would be a better option? Is it all down to my ears, regardless of price point and extra features? Or should I just stick with my CXA60 until it breaks because it does as good a job as anything else that I can reasonably hope to find?

It's all so confusing! I can't home demo EVERY option that's out there!
 

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,598
Likes
25,500
Location
Alfred, NY
So, if I, your average guy with no technical know-how, who has a Cambridge Audio CXA60 driving a pair of Klipsch Forte IIIs, wants to choose a new integrated amplifier, am I decieving myself into thinking a Sugden IA-4 would be a better option? Is it all down to my ears, regardless of price point and extra features? Or should I just stick with my CXA60 until it breaks because it does as good a job as anything else that I can reasonably hope to find?

It's all so confusing! I can't home demo EVERY option that's out there!
I’d ask myself, what is it I’m trying to improve? If it’s “sound quality” and my current amp isn’t clipping, then I’d look elsewhere. If one amp has features I want and another one doesn’t, that would also sway me.
 

Ingres3225

New Member
Joined
Jul 27, 2020
Messages
4
Likes
4
I’d ask myself, what is it I’m trying to improve? If it’s “sound quality” and my current amp isn’t clipping, then I’d look elsewhere. If one amp has features I want and another one doesn’t, that would also sway me.

Thanks for the answer!
When you say 'look elsewhere' do you mean 'find another amp' or 'look elsewhere in my system because, if my amp isn't clipping, then the amp is not the problem'?
 

SIY

Grand Contributor
Technical Expert
Joined
Apr 6, 2018
Messages
10,598
Likes
25,500
Location
Alfred, NY
Thanks for the answer!
When you say 'look elsewhere' do you mean 'find another amp' or 'look elsewhere in my system because, if my amp isn't clipping, then the amp is not the problem'?
Hah, that was ambiguous! Elsewhere in your system.
 

Ingres3225

New Member
Joined
Jul 27, 2020
Messages
4
Likes
4
Hah, that was ambiguous! Elsewhere in your system.

Great- that’s what I needed to know- I’m constantly seduced by beautiful, chunky amps which promise to transform my listening experience but I want to believe that, aside from features and build quality, an amp is basically an amp! This complex thread has just about convinced me that I’m right and saved me £3000 or so!
 
Top Bottom