That being said, is there a technical case for scaling? Is it completely myth, or is there anything to support the idea that an amp produces more than just volume headroom?
Sorry... W.O.T. as one cannot explain with just a few popular buzz words.
It is just volume headroom, bias (not of the output stage itself
) and voltage sensitivity as well as the volpot taper and FR being changed by certain equipment being either caused by unusual high output resistance or roll-off in the audible range.
As an example an average sensitivity headphone.
To double the volume (without a hint of clipping) the output power needs to be 10x higher.
At background listening levels you need maybe 0.03mW. This needs to be seen in the proper perspective. You need 0.03mW PEAK SPL that isn't the same as 0.03mW average level. Average levels may well be 0.003mW. Not a single device has problems with that.
To go twice as loud the amp needs to be able to provide 0.3mW. This brings you in the very comfortable listening level range you can listen to for hours. This too is no issue and can be reached with any device.
To go twice as loud again and reach comfortable loud levels you need 3mW. These levels can easily be endured for a couple of songs and when one likes a song and turns it up a little this is where you are. Also reachable with as good as any device.
It's the reason why most people say their headphone sounds great from a phone and you don't need an amp at all.
Next level requires 30mW. This will get you to an actively loud listening level that will be uncomfortable when this is longer than one entire song.
You are already reaching 110dB peak SPL.
When you really want to be impressed by sound you dial it up just a few dB more and end up in the 100mW peak region. Hitting 115 to 120dB peaks. This is something people do at headphone shows and to see how great the sound is, especially the bass. One does this for maybe 30 seconds or so and then dials down or switches a song.
There's the power thing. Now the people that talk about scaling will always be at least above 10mW and most likely reach 100mW peaks. Peaks that need to be clean, no distortion.
One would say well 10mW is not much. Even my phone ca reach that. In 16 ohm you need 0.4V... yes your average dongle will reach that on a headphone with an average sensitivity with say 100dB/mW.
For 32 ohm you need 0.6V.. yes a phone can reach that (usually around 1V or so).
50 ohm = 0.7V
120 ohm = 1.1V (so starts clipping already)
300 ohm = 1.7V so a phone won't be reaching an uncomfortable loud level and bass will suffer first, sound turns nasty.
The problem here is that those devices won't reach heavily clipping levels. They are not just limited to 1V but won't go much louder than 1V
It will stop at 3mW in 300ohm which is only a comfortable loud level. When you want to reach impressive levels you will need more.
So an amp is essential. No impressive sound means underwhelming experience. Add some more power and the impressive sound comes around. It 'scales' with amping. Need even more ? needs an amp with higher voltage.
Now.. most headphones will be around 100dB/mW but the impedance at this power efficiency level determines how much voltage is needed.
It's the voltage (combined with the impedance) that determines how loud it goes.
Then there is frequency response, bass heavy cans can have 10dB higher SPL in the lows (where impressive sound comes from) than rolled off headphones. 10dB requires 10x the power (3.1x the voltage). So bass shy headphones need about 10x more 'power' than bass heavy headphones to sound equally 'impressive' as sensitivity is measured around 500Hz to 1kHz in general.
So the bass shy versions need more 'power'. So they 'scale' more than bass heavy headphones.
Add to that... a lower sensitivity headphones can be as low as 80dB/mW and high efficiency ones as high as 115dB/mW.
For lower efficiency ones (plenty of those around) you thus need a lot more power than 10mW to become even remotely impressive.
These headphones will all sound fine on a phone at 'normal listening levels' but won't reach impressive levels.
So there you have the complexity.. efficiency, impedance and frequency response explained.
Such as in the case of impedance curves...would there be any reason to believe a high-impedance driver would sound closer to the "intended" FR with more power behind the same listening level?
Impedance
curves have nothing to do with it. In general the higher the impedance the lower the voltage sensitivity (see above).
I also mentioned volpot taper and unusual high output resistance.
There is this weird phenomenon that when one listens to the same SPL level but has the volume control at position 10 o'clock the impression you get is that you have LOTS of headroom while when your volume control is at 4 o'clock people get the impression the amp is 'whimpy'.
The 10 o'clock one can be turned up further (has more gain) and one is certainly going to do that or has done that and reached impressive levels.
Can't do it with the 'whimpy' one. Even when both are used at the same level people still feel the sound from the amp with higher gain is more 'impressive'. This too adds to the idea of scaling.
As most folks have no idea how much power they actually use, how much gain influences the impression of amps, what's the difference between low and high impedances, the voltages they need, the differences in efficiency, the influence of frequency response, the influence of hearing (equal loudness contours), the influence their surrounding has (a headphone demo in an audio fair or quiet at home in the evening) as well as other contributing factor the 'myth' is easily born. It is a technical thing in the end misinterpreted by folks that 'heard' things themselves and underestimate their level of comprehension.