Designing proper tests is a whole profession unto itself in the scientific world. There are books on the topic if you are interested. It's not just a matter of setting up a control and and experimental group. From what I've read, pretty much everything talked about in audio forums as scientific testing doesn't qualify.
How would you know this unless you are an audio scientist/engineer?
But yes, there are "books" and importantly, a number of published papers on how to test DACs. I have read them all and importantly since this is an interest of mine, both professionally and personally, I have the equipment and experience to put them to practice.
Importantly, I understand the architecture of the devices under test and their electronic design. This enables me to test the hypothesis they put forward, without any verification or data I might add, and use industry standard test and professional equipment to verify the same.
At the risk if being immodest, what I write is not some hobby work. I bought one of the first Audio Precision Analyzer while I was at Sony in early 1990s. At the time HP (now Agilent/Keysight) owned the measurement/instrument market for such devices. I was examining their measurement gear when I ran into the then new start up Audio Precision that had built and instrument that went miles farther than HP had. I was managing the design of a new "workstation" and we had a major audio subsystem and I wanted to make sure it performed. In that regard, I bought the first Audio Precision analyzer at Sony.
Fast forward to when I was working at Microsoft. To incentivize PC makers to build better quality audio subsystem, Microsoft created the WHQL program where minimum measured performance was required to get credit towards their Windows OEM license. My team managed WHQL and we worked hard to raise the bar in what was atrocious standards to something half defensible. So I purchased another Audio Precision Analyzer for the team. Indeed every PC OEM would purchase an Audio Precision analyzer to show conformance with WHQL standards.
I am also an electrical engineer, both as a hobbyist dating back to 1960s and professionally (Electrical Engineering degree in 1982). And I own this $25,000 instrument personally now.
So this is not some random data you are objecting to. My measurements have been published in magazines and reviewed by countless people in the industry and research. Not once has someone reached out to me/my editor saying the measurements are wrong.
Now, mistakes happen and I love to hear about them. But please don't just create doubt about my work with nary a reason to do so other than "there are books on scientific measurements." What are those books and how do they dispute my work?
For a decade also, my signal processing team at Microsoft produced technology that was verified in controlled listening and viewing tests prior to approval. Every time you buy a Blu-ray product or practically any product that plays audio sans apple that doesn't use/implement that technology. Those companies didn't come along because they liked Microsoft. They came along because we showed we knew what we were doing against competing solutions.
So whether it is objective measurements, technology solution or subjective testing, this is what I have done professionally and personally for decades.
I hesitated to write this but then yesterday in the process of researching another product I have under test, I land on this post of yours on CA forum:
Why do you say stuff like that? What knowledge do you personally have as expressed above that I don't know how to measure correctly?
I know you don't like the data but that should not rise to creating such doubt with no data of your own.
Let the information sink in please. The world is not as you have been led to believe. I can bring an army of researchers and audio scientists that would laugh at my attempt to even measure this device let alone believe its claims as you do. We are not all idiots as compared to a bunch of users who believe the techno-marketing statements of this company which they themselves have not bothered to verify.
Good grief
.