- Joined
- Aug 1, 2019
- Messages
- 126
- Likes
- 109
I am happy to put forward $1000 reward for him to show that a 60 dB SINAD amp sounds different than a 100 dB one in controlled, double blind test that is recorded for us to see. We have others that can help raise this bounty if needed. I am that confident he can't tell the difference let alone have a reliable preference.
Nelson Pass 350
@solderdude is referring to their monitoring system for AB comparisons.Sorry @solderdude , their preamps do not distort
https://www.mil-media.com/HV-3D-8.html
They are probably the best mic amps you can get, from anywhere.
Yeah I know he spoke about Pass. Isn't that interesting? Audio is not a simple thing at all, that could be wrapped in few numbers. I have similar experience, two almost no distortion components distinguishable through much worse parameters power amp and speakers, in A/B and even abx, sometimes. Let's not oversimplify. And I have personal experience with Millennia preamps, hats off.@solderdude is referring to their monitoring system for AB comparisons.
Earthworks has a comparable product, by the way: https://earthworksaudio.com/products/preamps/zdt/zdt-1022/
An improvement would be to design a system or application which works like foobar's ABX comparator, which calculates the result in the end:Let me describe our current AB technique (we generally don't do ABX -- I don't think it's necessary for valid perceptual testing at the individual, non-peer-review level). If you think our test is valid, that's great. We'll go it. If you think it's invalid, then perhaps you can help us with any scientifically fatal errors you perceive. You've set yourself up as an AB authority here. I do not make that claim.
These days, I no longer use analog tape. We use a DAW called Sequoia (we license our bit-reduction software "POW-R" to Sequoia, but that's another conversation). We keep our reference program on DAW (if there's interest, I could put our reference reel on a Dropbox). We use both headphone and room speakers for monitoring. We have a world-class reference room designed by George Newburn and built from the ground up with 9" concrete walls, wall / ceiling decoupling, around 15dB SPL, flat to 24Hz, ideal RT60 (Dunlavy SC5 + Nelson Pass 350). We monitor both mono and stereo sources. We use three different AB paths, depending on the type of circuit we're testing -- DAC, ADC, or pure analog path. We do both reference-to-DUT testing and DUT-against-DUT testing, the latter being qualitatively more difficult.
Our reference material is recorded in 10s snippets, each of which repeats over and over about 10X. We've chosen reference material that we have come to know intimately. The material was chosen for an ultra-wide range of sonic gymnastics: wide spatial, narrow center, fast transients throughout the spectrum, ultra-low freqs, ultra-high freqs, high DR, low DR, voice, pop, acoustic, big band, large hall ambience, dry room, dense, sparse, and so forth. We have around 25 different program sources, but only tiny, repeating snippets of the most critical aspect of each program are employed.
Each snippet is repeated over and over so that the brain can begin to focus on just one very particular sonic aspect. AB snippets are adjusted for better than 0.02dB level matching (which is challenging). Once the listener has suitably focused, the AB selection begins. We use the DAW's "solo" and "mute" functions to facilitate a self-blinded AB selection, using mouse clicks. The mouse is clicked repeatedly until the user does not have a clue if the source is "A" or "B". My personal method is to click the mouse rapidly while counting out numbers randomly. After 3-4 seconds of that, there is zero self-correlation. I guess you could argue otherwise. Until you try it, I suggest suspending judgement. It works. It's random.
But first we do it sighted. The user clicks back and forth, between reference and DUT (or DUT-recorded program, or DUT-DUT), to simply detect any timbre shift, transient variation, etc.. If a difference is detected sighted, then we go blinded. If detectable differences remain, we try to consistently identify a specific tell and stop there. We look at the screen and note the track. Wash, rinse, repeat.
If our "tell" is identified 8/10, we consider it a positive, and do it again. Another 8/10 is qualified. It means that the DUT is not giving us a correct image. Anything less than 2X 8/10 is a null.
Then we move to the next program material and do it all over again. We do this throughout the entire program cycle. What's interesting is that different types of DUTs may perform perceptually identical for most of the program sources, but when you get to that one "particular gymnastic routine" -- a tell is detected. This is why it's important to use the broadest range of program material possible, covering every aspect of sonic hurdle a circuit will be asked to deliver.
Wurcer mentioned doing AP tests of any DUTs in advance, which is of course de rigueur (without it, the test is effectively invalid and a waste of time). And I'm happy to have the "neutral party" clicking the mouse, in fact I would love that. It's a bother. Beyond that, I think our test methodology is scientifically sound.
Thoughts?
Well, I really don't think we should be quoting personal experience as evidence (unless we have evidence for that experience). It's not gospel either that the power amp and speakers will swamp other components since the spectrum of each has to be taken into account. But then if as you say your equipment had vanishly low non-linear spurs, then either level, noise or an issue with FR could be responsible. What else is there?Yeah I know he spoke about Pass. Isn't that interesting? Audio is not a simple thing at all, that could be wrapped in few numbers. I have similar experience, two almost no distortion components distinguishable through much worse parameters power amp and speakers, in A/B and even abx, sometimes. Let's not oversimplify. And I have personal experience with Millennia preamps, hats off.
I have tried it. Many times.Until you try it, I suggest suspending judgement. It works. It's random.
Oh? All of a sudden you got results of 8 out of 10 right as I suggested? As I said above, I have done this kind of testing. But never do I keep track of the outcome and get to 8 out of 10 right this way. That requires note keeping that is absent in your description.But first we do it sighted. The user clicks back and forth, between reference and DUT (or DUT-recorded program, or DUT-DUT), to simply detect any timbre shift, transient variation, etc.. If a difference is detected sighted, then we go blinded. If detectable differences remain, we try to consistently identify a specific tell and stop there. We look at the screen and note the track. Wash, rinse, repeat.
If our "tell" is identified 8/10, we consider it a positive, and do it again. Another 8/10 is qualified. It means that the DUT is not giving us a correct image. Anything less than 2X 8/10 is a null.
And hence the reason a) you will fail the test I am suggesting and b) puts doubt that you can reliably hear any of the stuff you have been reporting.I've made no claims to being an expert listener, or even that my AB testing is accurate.
You know how many stories there are in audio? Millions. It never means anything. People don't know how to perform proper, controlled testing of audio. They get something new to test. Their brain works differently now, focusing on what the new device is doing. And Bam! They hear more detail. More air. More nuance. More musicality. Not because the near gear did any of that. But because they are now listening differently.Around this time, my friend who recorded the San Francisco Symphony (Jack Vad) heard about my micamp design. He asked to borrow some channels, so I lent them. He called back a couple weeks later, asking me to build him 8 channels.
Oh? All of a sudden you got results of 8 out of 10 right as I suggested?
My sense is that all you have done is play A then B and go back a couple of times and call it done.
People don't know how to perform proper, controlled testing of audio ... And hence the reason a) you will fail the test I am suggesting and b) puts doubt that you can reliably hear any of the stuff you have been reporting ... If you are hearing such effects, it is likely due to frequency response errors or other simple things that we can readily measure ... Not sure how to advance the conversation anymore.
On second thought, just post some ABX tests of 320 kbps MP3 versus original for us. It will be a highly sobering experience if you have not been through it. Here is an example for you:
If the distortion is fully concentrated in a discontinuity such as crossover distortion IRCC Earl Geddes found this level easily audible.
Of course when one monitors using a Pass amp (which could have relatively high 2nd harm. distortion it could be really difficult to test for low distortion levels (say below 0.05%)
8/10 x2 is our standard pass/fail. Has been for probably 2 decades. This is also a standard threshold in ABX testing literature, as you know (often 3x @ 8/10 is preferred). What are you inferring?
If you dig back a few days, I mentioned this "8 of 10" metric before you brought it up. Here's the conversation,
"If we can detect a difference in 8 of 10 trials, it is a positive. The harder part is determining if the difference is better, worse, different, or a partial mix. It's hard work, as you know. "
Huh? Your "sense?" That's not science . That's belief. What I've described is our AB method. If you "sense" that I'm not being honest and forthright, then what's the point here? Why bother?
I have no way to respond to this. If you already know all these things, then why bother? A perplexing conversation. You seem to know so much about my future. (emphasis mine)
I could post 2 sound files from a recent micamp electrolytic coupling capacitor listening test (-125EIN / -115THD). One file would the worst of the bunch (10/10) and one would be the best (the series capacitor we selected for the new product). The third file would be the original reference. These are 20 minute files, 48 / 24 IIRC. Could you personally detect the differences? Have no idea. And if you -did- detect a difference, based on the apparent lack of trust in these recent responses, would you trust it?
This is why a fresh, on-site, real-time, neutral observer test is a better solution. I'm willing to do it in our lab, with our methods (which you've not yet shown to be invalid in any way).
Upload the files, please. The more examples with very low distortion but still audible differences, the better. I assume they are all using the same test signal?
Yes, identical reference program on all files. One is raw ref. The other two are thru identical HV-3 micamp (40dB gain, IIRC), except just different electrolytic capacitors in the front-end series path. (we source a high quality film cap that - in this application - sounds even more accurate than the best electrolytic, but in this new product it simply doesn't fit. The thing is HUGE). Keep in mind, the sonic difference between the best and worst is very subtle, and not something the casual listener would detect. It takes time and practice and patience.
And this brings up an interesting question. If the worst sounding cap and best sounding cap (assuming we detect a true difference) do not impact THD or noise (SINAD) in any way, then what is causing the detected timbre shift? The answer, I think, begins with a reflection on transient (one-off) distortions vs. periodic (static) harmonic distortions. But that's a conversation for another time.