Audyssey's Next Generation of Room Correction (MultEQ-X)

[peng]

We all understand that incorrect position. There is no miscommunication. Have you tried taking confirmatory measurements properly - same area, random sample points?

Incorrect position, aping measure point? He's only talking about an apples to apples comparison in a specific sense. You are reading into something that is not in that one sentence I referred to. He is not talking about whether the OP ran Dirac and Audyssey using the correct mic positions or not, he is only asking that if the OP is to use REW to measure the actual FR after running each of Dirac and Audyssey, then use the same mic positions (when measuring with REW/mic). There is no aping, he's not the one who place the mics during the OP's runs of DL and Audyssey.

I am not getting into debating what is "properly" with you. But yes, have tried all sorts of mic positions, angles, random sample points, have taken numerous REW graphs, never actually took a count but probably hundreds. Have only used Audyssey and DL standalone beta.

Have you read the Geddes and Blind sound power measurement paper?

Have read many of Geddes. Don't see any of your questioning being relevant anyway.

 

[Chromatischism]

For the sake of following your logic let's assume here we are testing a room EQ system which is based on a single sweep. Would you, in that case, suggest we test how well it has done it's job using a single sweep from the same point or using a spatially averaged measurement over the area of the listening position?

You missed where I said we're talking about different things. I was only responding to the checking of the "after" graph that Audyssey displays in the software.

 

[jhaider]

...if the OP is to use REW to measure the actual FR after running each of Dirac and Audyssey, then use the same mic positions (when measuring with REW/mic).

I'm not sure why this point is so hard to grasp. It should be clear to any reader from my previous posts that the above is exactly what I've been saying is pointless and time-wasting, while providing inferior information compared to a random sample of points within the same area of the room (aping the same points you use for calibration in confirmatory measurements tells you nothing about the stability of the approach).

I'm not writing anything about calibration procedure, only describing a superior - and easier! - procedure to take confirmatory measurements post calibration. Confirmatory measurements using the moving-microphone technique to cover the same area is another reasonable approach. But convincing people to do a lot of additional work by telling them they need to ticky-tack match the positions fed into the software in the first place when they take confirmatory measurements both wastes that person's time and gives her an inferior end result.

Have read many of Geddes. Don't see any of your questioning being relevant anyway.

Read or re-read the Geddes and Blind paper and you'll see what I mean.

 

[chych7]

Alright folks, some quantitative comparisons of Audyssey XT32/MultEQ-X (Denon X3600) vs. Dirac Live (PC, stereo trial license), for speakers only (Philharmonic BMR). I did not include subwoofers in this comparison because Dirac from PC doesn't have direct access to the subs, so it wouldn't be a good comparison. I use MSO/miniDSP for subwoofer EQ anyway, so personally I'm not that interested in Dirac vs. Audyssey for that part. I also have a fairly well treated room (around 16x 2'x4' mineral wool bass traps/panels and various diffusers), so the amount of room correction the software needs to do is probably less than most rooms.

Method
Dirac Live was calibrated with a UMIK-1 using the tight focusing (9 point) mic pattern; the "measurement cube" length was around 2 ft. Audyssey was setup using the same measurement points with the Audyssey mic (there will be variation in exact mic positioning though). For the target curve, I used a -1 dB/dec target for both Audyssey and Dirac (tilt used in MultEQ-X, and the control points were adjusted in Dirac to match). MRC/DEQ is off in Audyssey. 30 Hz cutoff frequency was applied in Audyssey (around where the BMRs roll-off), with the Dirac target adjusted to achieve this. I did not put any additional bass boost.

Edit: UMIK-1 was used in the 90 deg orientation and its associated calibration file for Dirac and post measurements.

Measurements
I did a 6 point measurement with REW at positions within the 9 point setup pattern (in about a 1.5 ft "bubble"), essentially around where my head would go. For each position, I measured Audyssey/Dirac/Uncorrected before moving to the next position. Var smoothing was used in REW, which doesn't smoothen the bass but smoothens the mid/highs. Postprocessing was done using Matlab to do the averaging/analysis/plot generation.

Feel free to recommend different ways or other ideas to analyze this data; a lot of you folks are much more experienced than me on this.

SPL vs. Frequency
Bass mode correction: Both Dirac and Audyssey do a similar job repairing room modes (below 500 Hz). Dirac is a few dB quieter than Audyssey by default.

Adherence to target curve: Dirac may be a little closer to the target curve, but keep in mind the mic I'm using for verification is the same mic used for calibration. Audyssey necessarily uses a different mic, so this is a clear inconsistency. Audyssey seems to boost the highs a little bit, and is more similar to the uncorrected response.

Left/Right Speaker SPL Error vs. Frequency

The better the left and right speakers are matched, the better the imaging (correct me if I'm wrong on this assumption!). Below are plots comparing the left and right speaker response, averaged over the 6 measurement points. I also computed the error between right and left speakers.

Both Audyssey and Dirac do a good job at reducing the Left/Right mismatch, about halving the maximum error. Audyssey appears to be slightly better, but this may be within the noise.

Left/Right Phase Error
Dirac is advertised as improving speaker phase, so I thought a measurement like this could capture it. As I understand, humans cannot hear absolute phase, but relative phase is detectable; reducing phase mismatch between the left and right speakers can improve imaging. In the below plot, the absolute value of the left/right phase error (averaged across 6 positions) is presented, as well as the average error as a single metric for comparison.

Both Audyssey and Dirac reduce the L/R phase error, but interestingly, Audyssey appears to be doing slightly a better job. Dirac looks to be doing something strange in the mid frequencies. Not sure what else to make of this.

Subjective Impressions
Very briefly - I am finding that the Dirac response sounds a bit flat, and I prefer it off for the mid/high response. Audyssey sounds better on than off full range. This is probably due to the difference of the target curves, with Audyssey resembling the natural response of my BMRs more, which naturally has elevated highs (see Erin's measurements). The extra boost in the upper freqs for Audyssey actually sounds nice to me, adding a bit of "sparkle" and soundstage. My hearing isn't that great up there so this helps. Bass sounds improved for both correction schemes.

My next plan is to change the Dirac target curve to be more similar to the Audyssey response (making it a more fair comparison), and try some blind tests with my wife as the subject.

 

[Chromatischism]

Alright folks, some quantitative comparisons of Audyssey XT32/MultEQ-X (Denon X3600) vs. Dirac Live (PC, stereo trial license), for speakers only (Philharmonic BMR). I did not include subwoofers in this comparison because Dirac from PC doesn't have direct access to the subs, so it wouldn't be a good comparison. I use MSO/miniDSP for subwoofer EQ anyway, so personally I'm not that interested in Dirac vs. Audyssey for that part. I also have a fairly well treated room (around 16x 2'x4' mineral wool bass traps/panels and various diffusers), so the amount of room correction the software needs to do is probably less than most rooms.

Method
Dirac Live was calibrated with a UMIK-1 using the tight focusing (9 point) mic pattern; the "measurement cube" length was around 2 ft. Audyssey was setup using the same measurement points with the Audyssey mic (there will be variation in exact mic positioning though). For the target curve, I used a -1 dB/dec target for both Audyssey and Dirac (tilt used in MultEQ-X, and the control points were adjusted in Dirac to match). MRC/DEQ is off in Audyssey. 30 Hz cutoff frequency was applied in Audyssey (around where the BMRs roll-off), with the Dirac target adjusted to achieve this. I did not put any additional bass boost.

Measurements
I did a 6 point measurement with REW at positions within the 9 point setup pattern (in about a 1.5 ft "bubble"), essentially around where my head would go. For each position, I measured Audyssey/Dirac/Uncorrected before moving to the next position. Var smoothing was used in REW, which doesn't smoothen the bass but smoothens the mid/highs. Postprocessing was done using Matlab to do the averaging/analysis/plot generation.

Feel free to recommend different ways or other ideas to analyze this data; a lot of you folks are much more experienced than me on this.

SPL vs. Frequency
Bass mode correction: Both Dirac and Audyssey do a similar job repairing room modes (below 500 Hz). Dirac is a few dB quieter than Audyssey by default.

Adherence to target curve: Dirac may be a little closer to the target curve, but keep in mind the mic I'm using for verification is the same mic used for calibration. Audyssey necessarily uses a different mic, so this is a clear inconsistency. Audyssey seems to boost the highs a little bit, and is more similar to the uncorrected response.


View attachment 188953


Left/Right Speaker SPL Error vs. Frequency

The better the left and right speakers are matched, the better the imaging (correct me if I'm wrong on this assumption!). Below are plots comparing the left and right speaker response, averaged over the 6 measurement points. I also computed the error between right and left speakers.

Both Audyssey and Dirac do a good job at reducing the Left/Right mismatch, about halving the maximum error. Audyssey appears to be slightly better, but this may be within the noise.

View attachment 188955


View attachment 188956

View attachment 188957

Left/Right Phase Error
Dirac is advertised as improving speaker phase, so I thought a measurement like this could capture it. As I understand, humans cannot hear absolute phase, but relative phase is detectable; reducing phase mismatch between the left and right speakers can improve imaging. In the below plot, the absolute value of the left/right phase error (averaged across 6 positions) is presented, as well as the average error as a single metric for comparison.

Both Audyssey and Dirac reduce the L/R phase error, but interestingly, Audyssey appears to be doing slightly a better job. Dirac looks to be doing something strange in the mid frequencies. Not sure what else to make of this.

View attachment 188960


Subjective Impressions
Very briefly - I am finding that the Dirac response sounds a bit flat, and I prefer it off for the mid/high response. Audyssey sounds better on than off full range. This is probably due to the difference of the target curves, with Audyssey resembling the natural response of my BMRs more, which naturally has elevated highs (see Erin's measurements). The extra boost in the upper freqs for Audyssey actually sounds nice to me, adding a bit of "sparkle" and soundstage. My hearing isn't that great up there so this helps. Bass sounds improved for both correction schemes.

My next plan is to change the Dirac target curve to be more similar to the Audyssey response (making it a more fair comparison), and try some blind tests with my wife as the subject.

Nice work, it is this kind of thing we need to see more of here on this site.

I have to assume that Dynamic EQ was disabled for these measurements?

 

[Keened]

Good work!

The only other thing I would try to measure would be the impulse response since that's another thing Dirac says it works on (and is maybe the thing their whole name is on? idk, I just thought it parsimonious that this came up when I was looking to see if the impulse response is related to the frequency response; it appears to be in at least a unidirectional relationship of impulse -> frequency but I don't understand enough of the math to say if you can do it in reverse)

 

[Chromatischism]

Good work!

The only other thing I would try to measure would be the impulse response since that's another thing Dirac says it works on (and is maybe the thing their whole name is on? idk, I just thought it parsimonious that this came up when I was looking to see if the impulse response is related to the frequency response; it appears to be in at least a unidirectional relationship of impulse -> frequency but I don't understand enough of the math to say if you can do it in reverse)

Not smart enough either but here is how you go from Frequency to Impulse and vise versa: https://en.wikipedia.org/wiki/Fast_Fourier_transform

 

[chych7]

Yes Dynamic EQ (DEQ) was off.

REW does compute impulse response. Below are my results (note: I'm not that familiar with impulse response in REW, not sure if I did this correctly). The right channel clearly shows improvement in Dirac... but Audyssey shows it too! Very interesting. Dirac and Audyssey are very similar.

The impulse response behavior should be indication of transient accuracy and response time of the signal. Faster decay time would lead to more accurate/sharper/clearer sound. Now, how much this is audible is another matter.

 

[chych7]

Ah I spoke too soon, the previous results used 10 Hz - 20 kHz frequency sweeps, so probably the impulse response is mostly dominated by bass/room issues. Below are plots from 500 Hz - 20 kHz. Now we see more of a difference between Audyssey and Dirac. Dirac seems to have better phase control, but Audyssey still is improving the impulse response.

 

[chych7]

Update: did some subjective assessments with my wife (who is not an audiophile) in a single blind manner to compare Dirac and Audyssey.
First, I matched the target curve in Dirac so it matches Audyssey. Some difference in the upper frequencies, but this is the best I could do (just measured at the MLP).

Since it would take too much time to switch between Dirac and Audyssey, I instead set it up such that the comparison would be: room correction off vs. room correction on. I didn't let my wife know when it was on vs. off or what method she was evaluating, which is the extent of the blindness of this test.

Dirac on vs. off: Slight preference for Dirac being on vs. off. Dirac sounded fuller in the bass and a little more expansive, but the mid-range/vocals sounded damped out. With room correction off, vocals sounded more clear and the sound seemed sharper, but not in a bad way.

Audyssey on vs. off: Strong preference for Audyssey being on. Audyssey widened the soundstage and made things clearer and more immersive, without degrading the sound in any way.

My own non-blind assessment mirrors her opinions. Dirac tended to make the mid-range sound a little muffled, whereas Audyssey didn't negatively affect it. I did find that Dirac improved the imaging precision of vocals and instruments, but it did so at the expense of reducing apparent soundstage. Audyssey on the other hand expanded the soundstage (by a couple of meters if I had to quantify it), didn't affect the tonality, and may have slightly improved imaging precision.

In the end, my preference is for Audyssey over Dirac for stereo music (full range correction). Limiting correction to <500 Hz seems to be a common thing for Dirac; I noticed that Amir does that in his own system. I didn't find any significant difference in bass correction for Dirac and Audyssey, as the frequency plots showed.

 

[Chromatischism]

It is my understanding that impulse response and frequency response are the same thing calculated in two different ways (via Fourier Transform). Imagine people's shock when you tell them that most room correction systems adjust the impulse, even if they don't advertise it.

Where Dirac goes further is to use (apparently) all-pass filters to adjust phase. Most reputable sources state that we can't hear phase itself so it is going to be down to whether other relationships altered by a change of phase are perceptible.

I'm very curious to know what is causing your perception of a larger soundstage. The frequency response is similar but there could be enough differences at specific treble regions to account for it, though I'm surprised by the magnitude you're describing.

 

[thorvat]

You missed where I said we're talking about different things. I was only responding to the checking of the "after" graph that Audyssey displays in the software.

Simple fact is that verifying the results of room EQ should be done with spatially averaged measurement no matter how particular room EQ system collects its data.

 

[thorvat]

It is my understanding that impulse response and frequency response are the same thing calculated in two different ways (via Fourier Transform). Imagine people's shock when you tell them that most room correction systems adjust the impulse, even if they don't advertise it.

Where Dirac goes further is to use (apparently) all-pass filters to adjust phase. Most reputable sources state that we can't hear phase itself so it is going to be down to whether other relationships altered by a change of phase are perceptible.

Some of us learned that during 2nd year of college while other's will certainly be surprised. It seems impossible for folks from both groups to participate in this discussion on equal basis.

There are room EQ systems that do far better job than DIrac with phase correction. Btw, when you alter the phase with FIR filters no "other relationships" beside phase are altered.

 

[peng]

I'm not sure why this point is so hard to grasp. It should be clear to any reader from my previous posts that the above is exactly what I've been saying is pointless and time-wasting, while providing inferior information compared to a random sample of points within the same area of the room (aping the same points you use for calibration in confirmatory measurements tells you nothing about the stability of the approach).

No, your point is not hard to grasp, but yet you are assuming that I found it hard to grasp. The fact is, the OP told you already you were talking about different things. Yet instead of explaining clearly your understanding of his point and your point, you keep insisting you were talking about the same thing. This is not the first time I see you are very keen on showing how knowledgeable you are (and I think you are) in this field, yet not so keen on trying to understand others points.

I only interjected (wish I didn't) when I too, felt you guys were talking about different things and you seemed upset about it for no reason. Noticed that I never said anything about your own points? So I am going to leave this alone and let Chromatishism continue if he chooses to.

 

[chych7]

I'm very curious to know what is causing your perception of a larger soundstage. The frequency response is similar but there could be enough differences at specific treble regions to account for it, though I'm surprised by the magnitude you're describing.

I was looking further into the REW outputs. RT60 times were similar, but C50 looked different and may explain what I was hearing.
You can see that No Correction and Audyssey are similar; Audyssey seems to be reluctant in changing the natural speaker response for mid-high frequencies. Dirac on the other hand significantly changes behavior above 2 kHz.

The C50/C80 plots represent system "clarity", which is the ratio of direct sound to reflected sound (REW explanation).

In the 2 kHz to 6 kHz, the Dirac C50/C80 drops. This may correspond to what I was hearing as vocals being less clear/more muffled.

Above 6 kHz, the Dirac C50/C80 increases vs. Audyssey, implying more direct sound and less reflected sound. These high frequencies are outside of the vocal range and the primary frequency of most instruments, so we may not be as sensitive to clarity. Instead, these frequencies contribute to soundstage from higher ordered harmonics. With Dirac, the reflected sound decreases, which can be perceived as lower soundstage (as per my subjective assessment).

Audyssey and no correction have similar response for upper freq for C50/C80, so why did it sound like Audyssey increased the soundstage? In this case, Audyssey also has an upper treble boost (vs. no correction), without affecting C50/C80, and may be the contributing factor.

Also to note - I looked at different target curves in Dirac, and they had similar C50/C80 response to the below plot. Dirac also seems to be slightly better at bass freqs.

 

[Chromatischism]

It shouldn't need to be said but just in case anyone joins here or finds this from Google, the results and conclusions seen above apply, at this point, to one pair of speakers and the room they occupy. It would be nice to see this kind of analysis applied more broadly, but it is a lot of work.

 

[jhaider]

Just getting around to fully digesting your very illuminating posts. Thanks for providing this data and showing your work!

Adherence to target curve: Dirac may be a little closer to the target curve, but keep in mind the mic I'm using for verification is the same mic used for calibration. Audyssey necessarily uses a different mic, so this is a clear inconsistency. Audyssey seems to boost the highs a little bit, and is more similar to the uncorrected response.

View attachment 188953

That is a huge and potentially dismaying surprise to me. Before further analysis, can you confirm your microphone orientations and calibration files applied? Namely, that if your microphone was pointed up, a 90deg (also called "grazing" or "diffuse field") cal file was used for Dirac calibration, as well as for your post-correction measurements of both systems? Here are two examples showing differences in cal curves between on axis (free field) and grazing (90deg/diffuse field) for two different microphones, iSEMcon and UMIK. The difference in cal files between from 2-12kHz are very similar to the level and directional errors between predicted and measured in your Audyssey data:

 

[chych7]

That is a huge and potentially dismaying surprise to me. Before further analysis, can you confirm your microphone orientations and calibration files applied? Namely, that if your microphone was pointed up, a 90deg (also called "grazing" or "diffuse field") cal file was used for Dirac calibration, as well as for your post-correction measurements of both systems? Here are two examples showing differences in cal curves between on axis (free field) and grazing (90deg/diffuse field) for two different microphones, iSEMcon and UMIK. The difference in cal files between from 2-12kHz are very similar to the level and directional errors between predicted and measured in your Audyssey data:

Yes I did use the UMIK-1 in the 90 deg orientation, along with its 90 deg calibration file, I should have mentioned that in the initial post. This way it would be "fair" with respect to Audyssey, but also my seat's headrest would be in the way, preventing measurement at where my head would be if I were to use the 0 deg orientation. I also briefly had two UMIK-1s, and compared the two. They were within 1-2 dB of each other for both orientations. Now with respect to the Audyssey mic, I don't know how to confirm its accuracy...

 

[jhaider]

Yes I did use the UMIK-1 in the 90 deg orientation, along with its 90 deg calibration file, I should have mentioned that in the initial post.

Well that sucks. I was hoping for a simple and easily correctable user error, rather than a defective room correction system. And it is an arguably disqualifying defect: you said zig (lower the HF) and Audyssey decided to zag (boost the HF). That's in essence, "eff the target curve you assign me, I'll do whatever the hell I want." Have you reported this bug to Audyssey? I would start there. It could be as simple as someone baking the wrong cal file into the program.

In the measurements I've done of Audyssey MultEQ XT32 (three different AVR/Ps, three different Audyssey mics, two different calibrated mics used for post-correction measurements) there was always very good agreement between predicted/presumed and actual. FWIW all three Audyssey mics that have passed through me in the last decade (two review units, one personal purchase AVP) have been just fine. However, I've also seen measurements from others suggesting more variance than in my sample of 3.

This way it would be "fair" with respect to Audyssey, but also my seat's headrest would be in the way, preventing measurement at where my head would be if I were to use the 0 deg orientation.

I also think grazing/diffuse field is generally the preferred way to take in room measurements.

I also briefly had two UMIK-1s, and compared the two. They were within 1-2 dB of each other for both orientations. Now with respect to the Audyssey mic, I don't know how to confirm its accuracy...

IF you have interest (regardless of the defect, you seem to be satisfied with the sound you’re getting, so I’d understand a lack of interest) one quickish way to see where Audyssey is defective is a rough microphone comparison: same speaker, swap mics keeping the capsule at same height, distance, and orientation. I don't think the Audyssey uses a weird pinout, so you should be able to plug it right into any computer or USB sound card mic in that accepts a standard 1/8" TS plug. Or just put them side by side if you have a stand that can do that.

You won't get a "perfect" calibration, but given the discrepancies here IMO rise to the level of defective product (both in level and, more concerning to me, directional, error) the data would be be good enough, and grounds for a microphone replacement under warranty if that ends up being your defect.

I hope it does end up being your mic or a simple bug such as Audyssey loading the wrong cal file. If MultEQ X doesn't work properly or transmit data to the AVR/P properly, that is...problematic.

 

[jhaider]

No, your point is not hard to grasp, but yet you are assuming that I found it hard to grasp.

I'm assuming nothing. I'm reading.

The fact is, the OP told you already you were talking about different things.

You two may think that, but the actual text on the screen says otherwise. I note your hand-waving is not accompanied by any actual explanation of said alleged differences in scope. So, here's your opportunity to do so: please explain to the group the relevant and material distinctions you see between the following statements:

Group 1:
C (quoted, emph. added):
"Often people want to compare their results to the Audyssey predicted response. THAT is when you have to use the exact same measurement points. People usually don't, then complain that Audyssey is "making things up". Hence, the disclaimer on the correct procedure."
C (quoted, emph. added) To clarify, this is "responding to the checking of the "after" graph that Audyssey displays in the software."
P (quoted, emph. added)
"he suggested using the same 8 points ref. post#413)...when using REW to find out how well each performs in terms of actual results (averaged) versus the corresponding target curves."

Group 2:
J (summarized):
There is no need to try to place the microphone at the same points used in calibration in order to confirm a room correction system's predicted results with an independent measurement system such as REW or FuzzMeasure. Instead, the proper approach is to sample random points in the same area covered by the calibration points. That way you can test both whether a RC system (a) offers a reasonable prediction of its end result and (b) results that are stable over an area.

To elaborate, here's some data showing RC predicted-vs-actual comparisons done properly:

• @chych7, post #470 (emph. added):
  • "Dirac Live was calibrated with a UMIK-1 using the tight focusing (9 point) mic pattern; the "measurement cube" length was around 2 ft. Audyssey was setup using the same measurement points with the Audyssey mic (there will be variation in exact mic positioning though) [JBH note - I don't think that's actually necessary, but can't hurt and will stave off dumb comments in forums.]...I did a 6 point measurement with REW at positions within the 9 point setup pattern
• Fig. 22 of my Denon X4100 review, "Audyssey Reference Displayed vs. Measured Inverse Curve" See text for methods.
• Fig. 23 of my Bryston SP4 review, "Dirac Live Auto-Target Fit." See text for methods.

Of note, both systems seem to be doing a reasonable job in my old setup of showing you what they do. As I wrote above, @chych7's measurements uncovered a defect in MultEQ X, so results obtained from basic Audyssey XT32 may not apply to MultEQ X. (A broken mic, if that ends up being the root cause of the defective calibration, is still an Audyssey MultEQ X defect in my view.

This is not the first time I see you are very keen on showing how knowledgeable you are (and I think you are) in this field, yet not so keen on trying to understand others points.

When you have the facts on your side, argue the facts.
When you don't have the facts on your side, argue the theory.
When you have neither the facts or theory on your side, lob condescending bullshit.