manisandher
Addicted to Fun and Learning
Priors:
1. For many years, I'd been certain that I could hear clear differences between bit-identical playback means (e.g., different USB cables, streaming vs. local playback, different software players and their bit-identical settings, etc.) - something that has bugged me no end over the years
2. I wanted to understand what could be causing what I was hearing, with the hope that an interesting insight (a way of further improving the SQ of digital audio perhaps?) might be gained
3. I was confident that I could demonstrate what I was hearing in a controlled listening test
4. I invited @mansr to my place (paid for his train ticket, no less) to help conduct a listening test (I chose 2 different but bit-identical setttings in a software player as the single variable)
5. the test was almost double-blind
- Mans controlled the test from my office, two doors and a corridor away from me in my listening room (I could not see or hear anything he was doing)
- Mans used a random generator on his phone to determine whether X=A or X=B
6. I was expecting to get close to 100% correct, so clear were the differences I'd been hearing for years
Likelihood:
- I scored 9/10 in the only ABX we conducted
- I got #9 incorrect - we were 15 minutes into ABX at that point, and I was feeling fatigued (we'd conducted 2 non-ABX tests beforehand - see below)
- input into DAC during ABX was shown to remain bit-identical*
Commoness:
- p = 0.01
Disclosure:
- we'd conducted 2 non-ABX tests before the ABX, where I scored no better than guessing
- even here, I was certain I could hear differences, but had no reference against which to assess, so simply guessed A or B in my responses
- I don't think it's fair to include the results of these non-ABX tests in the ABX results (also, I'd never been involved in any tests beforehand, and was adjusting to the strange/unnatural setup)
- the results of these two non-ABX tests suggest there were no 'tells' during any of the tests
- * there was a glitch in the auto-start function of the digital recorder being used, so the first few ms (out of ~15,000 ms) of each digital capture had to be ignored
- there were no conflicts of interest on my part - I was neither selling nor promoting any product/service (I'm not a manufacturer or fan of any particular product)
- this test was conducted out of pure curiosity on my part
- I HATE that these differences seem to exist (to me, at least) - I do NOT want USB cables or different software players to sound different... and yet they seem to (to me)
It's not inconcievable that someone selected randomly from the street would score 9/10 in the first ABX they sat - there's a 1 in 100 chance that they would.
However... given the Priors above, surely the 9/10 result shouldn't be easily dismissed as a simple fluke?
Could someone help me think this through using Bayesian reasoning please?
FWIW, I'm still certain I can hear differences in bit-identical replay, to my genuine chagrin. No more ABX tests for me though - I hated the whole experience. However, using a low-noise DAC-ADC chain and @pkane 's DeltaWave, I'm pretty confident I can measure these differences now. I intend to do just that over the Christmas break, and will come back with what I find.
Mani.
1. For many years, I'd been certain that I could hear clear differences between bit-identical playback means (e.g., different USB cables, streaming vs. local playback, different software players and their bit-identical settings, etc.) - something that has bugged me no end over the years
2. I wanted to understand what could be causing what I was hearing, with the hope that an interesting insight (a way of further improving the SQ of digital audio perhaps?) might be gained
3. I was confident that I could demonstrate what I was hearing in a controlled listening test
4. I invited @mansr to my place (paid for his train ticket, no less) to help conduct a listening test (I chose 2 different but bit-identical setttings in a software player as the single variable)
5. the test was almost double-blind
- Mans controlled the test from my office, two doors and a corridor away from me in my listening room (I could not see or hear anything he was doing)
- Mans used a random generator on his phone to determine whether X=A or X=B
6. I was expecting to get close to 100% correct, so clear were the differences I'd been hearing for years
Likelihood:
- I scored 9/10 in the only ABX we conducted
- I got #9 incorrect - we were 15 minutes into ABX at that point, and I was feeling fatigued (we'd conducted 2 non-ABX tests beforehand - see below)
- input into DAC during ABX was shown to remain bit-identical*
Commoness:
- p = 0.01
Disclosure:
- we'd conducted 2 non-ABX tests before the ABX, where I scored no better than guessing
- even here, I was certain I could hear differences, but had no reference against which to assess, so simply guessed A or B in my responses
- I don't think it's fair to include the results of these non-ABX tests in the ABX results (also, I'd never been involved in any tests beforehand, and was adjusting to the strange/unnatural setup)
- the results of these two non-ABX tests suggest there were no 'tells' during any of the tests
- * there was a glitch in the auto-start function of the digital recorder being used, so the first few ms (out of ~15,000 ms) of each digital capture had to be ignored
- there were no conflicts of interest on my part - I was neither selling nor promoting any product/service (I'm not a manufacturer or fan of any particular product)
- this test was conducted out of pure curiosity on my part
- I HATE that these differences seem to exist (to me, at least) - I do NOT want USB cables or different software players to sound different... and yet they seem to (to me)
It's not inconcievable that someone selected randomly from the street would score 9/10 in the first ABX they sat - there's a 1 in 100 chance that they would.
However... given the Priors above, surely the 9/10 result shouldn't be easily dismissed as a simple fluke?
Could someone help me think this through using Bayesian reasoning please?
FWIW, I'm still certain I can hear differences in bit-identical replay, to my genuine chagrin. No more ABX tests for me though - I hated the whole experience. However, using a low-noise DAC-ADC chain and @pkane 's DeltaWave, I'm pretty confident I can measure these differences now. I intend to do just that over the Christmas break, and will come back with what I find.
Mani.