I've heard the Accuphase you mentioned in person paired to a variety of loudspeakers at my local shop in Tokyo. It's a very versatile amplifier with plenty of damping factor which is ultra important to control the reverse pistonic movement of a loudspeaker. The build quality is typical Accuphase as in built like a tank. I also compared it to SPEC Corp, a japanese exotic class D amplifier whose design goal is to sound like a powerful triode in which they have succeeded. Link:
https://spec-corp.jp/index.html Their top model has a upwards of 98% efficiency from the AC mains to output power! They also use a design to control the speakers reverse cone movement to achieve a controlled sound of authority.
The Harbeth's you are running is a traditional design with modern drivers, I'm not certain if they still use bitumen for internal damping of the panels, but it's a fairly benign load for the amplifier. If you are unhappy with the amplifier, I would start to point fingers at the speakers over the amplifier.
I would urge you to find a SPEC Corp dealer.
https://www.widescreenaudio.be/product-category/audio/brands/ bring a record or any audio file, and give them a comparison. SPEC doesn't use Ncore/hypex/puri-fi amplification. They use a chip from International Rectifier Company, that has a smaller internal company called infeneon. They don't sound like class-d. Class D is a switching PWM amplifier, not digital. They use an awful lot of intelligent, interesting designs and bespoke parts on the insides to achieve the sound that they produce.
Either way you go, I don't think you can go wrong. Especially if you listen to anything with real instruments!
Enjoy your future purchase. You will not be unhappy no matter which direction you go.