I have an impression that mastering engineers design their own gear, use rare in-house devices or analogue gear, and yet start from scratch all the time when thinking about how a particular project can be tweaked to sound as good as possible with the minimum of intervention.
Some mastering eng. did design tools. Most don't. Why: you don't need it at all. Most plugins are a lot better than analogue gear. There is a lot
of mystic/vodoo around analogue gear. The last years, the differences have vanished. Most fan of analogue stuff don't want to do blind tests which gives you an idea. Analogue tools adding distorsions are useful. Having a tube EQ or a tube compressor can be heard (this requires concentration).
Many of them have authored books. Online courses to impart some skills have been created.
In the past, recording studios were far more popular but my impression of recording and mixing is that it is difficult and complex, and seems far beyond the reach of many casual listeners.
Part of what made mastering (vinil, tape) has mostly vanished. The part which is left is tuning the sound which is part art part science.
At this point, automated mastering services are appearing but regardless of that, many content creators will also opt to 'master' audio on their own.
As always, differences between automatic mastering (to a target) and manual one are decreasing. The automatic mastering services will be very very good soon and the only part left would be to define the target that match the recording (more art than science).
Is it possible for any audiophile to practice what a mastering engineer does? Can a person, with or without studio working experience, set up a good studio environment according to the books, use smaller but well-performing studio monitor speakers e.g. Adam Audio etc, and such technology as mastering software, plugins, studio monitor headphones etc to master their own streaming content?
Yes and no. The science part is easy to learn. Calibrating your room, using oZone smartly is not hard. Creating a sound signature that someone will recognize is something different. Also often the mastering eng. is asked what he/she can do with a bad mix. That requires some knowledge or a lot of time.
Aren't the audiophiles who consider things to the highest and deepest levels those who are basically mastering engineers even if they don't work professionally as mastering engineers?
Mastering: make a sound pleasant to a maximum of people
Audiophile: tune to my ears in my room.
Not exactly the same objective.
If this is the case, mastering engineers will quickly become unnecessary, since AI and amateurs can easily replace the occupation.
They are dying slowly. The added value of a mastering eng. exists only if 1. you are famous 2. you are very good at what you do.
Life is much harder if you have medium skills since AI is catching up.
It is arguable that digital plugins, with much trial and error, is capable of recreating what was only possible with analogue gear.
Sure, for a long time now.
It seems it takes more than just an audiophile to make a mastering engineer. ...
Mastering is a profession. Like many of them you need to learn and practice, it takes some time. There are plenty of way to learn
http://www.atiam.ircam.fr/en/ is a good one for example. IRCAM by the way has plenty of excellent courses
https://www.ircam.fr/transmission/formations-professionnelles/.
Overall, since mastering as a profession has become firmly established in the 20th century, the only way people can enter that field is by working at one of the already existing studios. Obviously such skills as vinyl authoring are only possible to gain there. In the digital realm, trial and error could get you to the same level perhaps, but that is limited by how much skill+experience you already have in working at a proper studio. Most mastering engineers would need to be in a studio affiliated with or directed by a larger corporation to remain in business without too much trouble.
vinyl authoring is dead (almost). post prod for movie is the future. Good time to learn ProTools or Resolve/Fairlight. Movies for example are done 100% in the box (ITB). 1 computer, 1 accelerator and you need nothing else.
But let's throw common sense out the window. Ignoring the need to have speakers which you understand deeply and can work by analogy to imagine even larger speakers, let me ask what DAC does a mastering studio use? Can Sonata HD Pro suffice? Can Audacity be mastering software? Can studio monitor headphones like Sony's M1ST, Yamaha's MT8 or Audio-technica's ATH-M series suffice? And ultimately, if most consumers use the most commonplace worthless gear to listen, isn't it proper for the mastering to take place using just a DAC and headphones? In most cases there'd likely be no issues with how it sounds on a radio, a TV, a car, or a high end setup, although one could simply test it out rather than rely on analogy and experience with professional studio speakers.
Most studio person I know obsessed about their gear. They will buy what magazine are pushing them to buy. Read a few threads on gearslutz and you will have an idea. This is very similar to audiophile behaviour.
Can every audiophile be a mastering engineer with only a Sonata HD Pro, a computer, and ATH-M40x?
People are limited by the time and motivation they want to put into mastering. If they really want to learn, you don't need the best tools.
Plus a 30$ dac is good enough if you are all in the box. Remember mastering you get a waw file, you produce a a CD or similar. Everything is digital. DAC is used once to listen to the music, no loop of AD/DA.