Multi-tone audio testing?

[Blumlein 88]

Another multi-tone comparison test.

This time, with the Holo Spring R2R DAC doing the D2A conversion and Apogee Element24 doing the A2D duty. Still at 24/96k:

1kHz single tone:
View attachment 60518 TD+N = -104dB

11kHz single tone:
View attachment 60519 TD+N = -90dB

CCIF 18.5k/19.5k two-tone:
View attachment 60520 TD+N = -90dB


32-tone test:
View attachment 60521 TD+N = -102dB

500-tone test:
View attachment 60522 TD+N = -79dB

500-tone test up to 20kHz:
View attachment 60523 TD+N = -103dB

20k-tone test, phase and frequency response:

Oversample (OS) mode:
View attachment 60527 TD+N = -65dB

Non-oversample (NOS) mode:
View attachment 60528 TD+N = -54dB --> 10dB worse than OS mode(!)

Phase:
View attachment 60525

Frequency response (kinda lumpy, isn't it?):
View attachment 60526

So what did you conclude if anything from these various multi-tone tests?

 

[RayDunzl]

So what did you conclude if anything from these various multi-tone tests?

Made me think of the Dubai Skyline

 

[pkane]

So what did you conclude if anything from these various multi-tone tests?

Still work in progress, so no major conclusions yet. Certainly there is some difference between a two-tone and, say 1000-tone test. The multi-tone result does show more clearly the entire frequency spectrum. The TD+N number isn't that much different between a few tones, and few thousand. But, a few thousand tone test signal also produces some interesting insights, such as phase and frequency response, as well as clock drift and delay. I'm pretty sure I can also derive linearity and jitter metrics from it, but have yet to do so.

In effect, the Multi-tone Analyzer tool can provide a quick way to get most of the same results that DeltaWave produces, but all in one step and with only a few seconds of loopback capture.

 

[bennetng]

How about bandpassed white noise? This site has such a test.

Some examples:
https://reference-audio-analyzer.pro/en/report/dac/schiit-fulla.php#srw43
https://reference-audio-analyzer.pro/en/report/dac/gigabyte-ga-a75-ud4h.php#srw43

Fulla Schiit's noise floor is obviously higher when the noise is playing, but the Gigabyte motherboard doesn't exhibit this issue.

 

[pkane]

Well if you are creating these multi-tones because people like to think it is more like music, how about having the level go down as the tone goes up. Something like a pink noise shape. Subtract out the tones and compare with what is left. Multi-tone music test.

Not quite music yet What I did was to randomize the individual tone amplitudes and frequencies in the same multi-tone test to see if the results would be different. Not much of an obvious difference when measuring Apogee Element24:

Previous test with the log-spaced tones at equal amplitude:

 

[RayDunzl]

@pkane

You keep testing some "good" piece of gear.

Don't you have a piece of junk with which to test?

 

[pkane]

@pkane

You keep testing some "good" piece of gear.

Don't you have a piece of junk with which to test?

I'll have to dig into my junk drawer

 

[pkane]

@pkaneDon't you have a piece of junk with which to test?

Here's a less perfect test: Creative Labs X-Fi 5.1, DAC line output measured by Apogee Element24 ADC, both @ 24/96k:

20000-tone test:

 

[Rja4000]

Is your TD+N value against the Peak or against the average of the signal ?

In other words:
Does your TD+N match this definition:

"Total Distortion + Noise (TD+N) is defined as the square root of the ratio of the
power of the total distortions plus noise to the power of all the fundamentals, in other
words, the square root of the ratio of the total power less the power of all the
fundamentals to the power of all the fundamentals."

 

[pkane]

Is your TD+N value against the Peak or against the average of the signal ?

In other words:
Does your TD+N match this definition:

"Total Distortion + Noise (TD+N) is defined as the square root of the ratio of the
power of the total distortions plus noise to the power of all the fundamentals, in other
words, the square root of the ratio of the total power less the power of all the
fundamentals to the power of all the fundamentals."

Yes, except the calculation of TD+N is done in the time domain, frequency domain is used to eliminate the fundamental frequencies.

 

[pkane]

By the way, thought it might be interesting to zoom-in on the difference in signal between what was captured, and then with the fundamentals removed. Here's the capture of Sound Blaster DAC, zoomed out, near the top of the frequency range (blue=captured, white=with fundamentals notched out):

And here's the area near the right of that plot, zoomed in. The white signal highlights distortions and noise:

 

[Rja4000]

Yes, except the calculation of TD+N is done in the time domain, frequency domain is used to eliminate the fundamental frequencies.

Just out of curiosity:
If you remove the fundamentals in frequency domain, what's the FFT size you use ?

For 20000 tones, you'd need quite a high FFT size, for the resolution to be narrow enough at 20Hz.

Except if you use linearly-spaced tones (Like every 1Hz), of course, but what would be the use of so many tones then ?

 

[pkane]

Just out of curiosity:
If you remove the fundamentals in frequency domain, what's the FFT size you use ?

For 20000 tones, you'd need quite a high FFT size, for the resolution to be narrow enough at 20Hz.

Except if you use linearly-spaced tones (Like every 1Hz), of course, but what would be the use of so many tones then ?

I originally started with variable FFT size, but quickly found that 1M worked best for large, high frequency resolution multi-tone tests. That's what I use now.

 

[Rja4000]

I originally started with variable FFT size, but quickly found that 1M worked best for large, high frequency resolution multi-tone tests. That's what I use now.

OK
1M FFT has a bin width of 0.045776Hz at 48kHz (twice as much at 96kHz)
If you have 20000 tones logarithmically equally spaced, spacing between 2 tones would be much closer than that around 20Hz..
???

 

[pkane]

OK
1M FFT has a bin width of 0.045776Hz
If you have 20000 tones logarithmically equally spaced, spacing between 2 tones would be much closer than that around 20Hz..
???

True. I simply don't use any fundamentals that would be closer than a few Hz from each other. This results in fewer fundamentals than the number specified (actually around 7k tones when 20k is specified ), but they don't overlap.

 

[Rja4000]

Now I get it !

 

[Rja4000]

Agreed. I implemented a simple crest-factor/phase optimization algorithm in the multi-tone generator. Varying the crest factor and also randomizing or re-distributing the tones in the frequency space could be an interesting comparison. Interpreting the results is really what I'm after with this tool.

That's one of the things that makes it so hard to have a single figure to compare those multi tone measurements:
Not 2 test signals are equal.

Would you mind sharing more about the way you optimize the crest-factor ?
I'm looking for that for some time and I'm not able to implement it.

 

[Rja4000]

32-tone test signal:
View attachment 60404 TD+N = -114dB

I have still a few questiions:

Let's compare how it looks like when I measure my RME ADI-2 Pro fs with the same settings.

Comparing your 31 tones here:

and mine:

I'd expect the TD+N figures to be more similar.

Yes, mine is only in 20-20kHz range.
But it actually looks better.
Peak level is -0.1dBFS, 4s window , 4 linear average

Same, but non averaged, computed on 10s window:

In my case, -108dB TDN is versus the stimulus power, ie versus the average rms level of each indivudual tone (-24dbFS).
So that's -132dBFS in this case.

Before you ask, there is no difference if I select 1M FFT size.
Just the kind of difference due to different crest factor, since I select phases randomly.

I know looking at graphs may be misleading, but if we could find a way to get results that we can compare, that would help, I guess.

Any idea why we see such a difference ?
Or how to make sure we can compare the figures ?

 

[Rja4000]

I also tried the 100 tones

Of course, there is an impact on crest factor (6dB)

And comparing to my Dell XPS 15's Realtec Output
we see a dramatic difference (as expected)

 

[pkane]

I have still a few questiions:

Let's compare how it looks like when I measure my RME ADI-2 Pro fs with the same settings.

Comparing your 31 tones here:

and mine:

View attachment 61431

I'd expect the TD+N figures to be more similar.

Yes, mine is only in 20-20kHz range.
But it actually looks better.
Peak level is -0.1dBFS, 4s window , 4 linear average

Same, but non averaged, computed on 10s window:

View attachment 61432

In my case, -108dB TDN is versus the stimulus power, ie versus the average rms level of each indivudual tone (-24dbFS).
So that's -132dBFS in this case.

Before you ask, there is no difference if I select 1M FFT size.
Just the kind of difference due to different crest factor, since I select phases randomly.

I know looking at graphs may be misleading, but if we could find a way to get results that we can compare, that would help, I guess.

Any idea why we see such a difference ?
Or how to make sure we can compare the figures ?

That's a great idea, I was hoping to find some comparison done by others!

A few suggestions, and I certainly welcome any ideas on what else we can try. My captures seem to be dominated by N and not distortion. So perhaps we should measure noise to compare.

1. Have you tried lowering the dBFS level from -0.1dB? At least with my equipment, that's too close to clipping. I usually back off by a dB or two just to make sure I'm not hitting the limit of the ADC. As you can see, for me the peak was -1.9dBFS.

2. Can you try measuring TD+N at very different dBFS levels? Say at -20dBFS and at -80dBFS? Just curious how the TD+N calculation in Virtinis may differ from mine. I get a fairly consistent result, down to about -96dBFS. Here are a couple of captures at -20, -80, and -96dBFS:




3. Does Virtinis display the crest factor? What was it for this test? For mine, it's about 4.4.

4. Was this A-weighted (TDN_A is what makes me think it might be)? If so, my result is even that much lower, since mine was unweighted. That would make it an even larger difference to explain away [edit: I see this is just channel A]

5. I can export the frequencies and phase for my test if there's a way for you to load them into Virtinis. Or, if you export yours, I can load it into my test. At least this will give us more of an apples to apples comparison.

6. Open to other ideas on how to validate

By the way, is there a trial download of Virtinis software available? Don't know much about this software. [edit: just downloaded trial. May need help to set this up the same way you have]