@MayaTlab
I would ask oratory if he'll measure your headphones and then measure them on your head when Harman EQ'd, and use that as some kind of reference. It won't be perfect, but it's better than nothing, and I don't see much variation between Harman EQ'd headphones on my head until about 7 kHz (at least for headphones that are not sensitive to seal).
That would rather be a study in sample variation then
. Which is absolutely a thing at least with some headphones.
But I've had the opportunity of measuring three Airpods Max on my head, both channels on the same ear (as they're nearly perfectly symmetrical front to back it's valid to do so, unlike other headphones, so that's basically six samples in total), and I'm getting fairly similar results on all three (and actually a near perfect match below 800Hz where the ANC circuit does its magic), so even with models with a low sample variation I'm not getting similar comparative results
on my head as Oratory's measurements.
That's two Airpods Max, two averages of five measurements each for the four cups in total on my right ear (eight traces in total then),
during the same measurement session with my DIY probe mic (which wasn't moved).
Given how much variation the APM produces above 1kHz with slight variations in pad compression (including in the ear canal gain region) I think it's fair to say that whichever sample variation there is, it's buried under the seatings variation "noise" (I haven't saved the individual seatings so the two averaged traces per cup will have to make due).
That, for example, is the typical sort of variation I see when applying slight pressure at the bottom rear of the cups and pulling the headband slightly to re-balance the pressure around my ears (the APM's default headband clamping force + pivot design applies uneven pressure around my ears) :
Note : this was during a different measurement session hence the slightly different absolute values (Slight changes in the probe's position affect the results increasingly so as frequency rises, which is why I prefer to only show comparative measurements during the same session).
So I'm quite confident that what I'm getting out of the Airpods Max is not a question of sample variation.
There's still the question of whether the results I'm getting with the probe are actually real, ie if it measures a 3-4dB difference between headphones A and B at 4600Hz, is it really the case ? Besides torturing the apparatus in various ways to test for issues that could compromise the comparative results (such as applying various degrees of pressure on it at various points or re-locating the way it's routed out of my ear to try to assess whether or not different headphones' pads are compressing it or not in different ways), this is what I'm spending most time now trying to characterise since I'm past the question of whether or not I can get reliably repeatable comparative results (at least with headphones with low seatings variations like my HD650 or HD560S). One way of doing so for starters, I think, is to look at extreme cases, and apply counter-EQ. So, for example, the Hi-X65's fairly strong response I measured at 6200Hz or so vs. the HD560S or HD650 (+5-7dB, which should be very audible). Or the Airpods Max's ear canal gain region, which is nowhere near as depressed on my head as on quite a few measurements available online, and to which Oratory's preset applies a broad +3dB adjustment, which should be
very audible. So far I'm starting to get quite confident that what I'm getting is really happening, to a certain degree (you won't see me use these in-ear measurements as absolute truth anytime soon or apply automated EQing based on these). Can I make two headphones sound perfectly identical ? probably not. But what I'm stating to think is that using this probe, I can make two headphones
with low seatings variation sound closer to each other than with third party measurements and I would have less confidence in my capacity to distinguish them (let's just say that I am very confident that I'd be able to distinguish my HD560S and HD650 even after applying third party profiles).
Regardless of my own amateurish experiments, that headphones on someone's head deviate
at least a little bit from dummy head measurements in ways that are unpredictably inconsistent from listener to listener is not something coming out of left field, it's been measured already several times and probably well-known to most headphones manufacturers.
It's highly likely that it's happening on your own head too to a degree
.
Sorry for the long deviation away from the PC38X.