That would be extremely useful. I can get a developer to do the plug-in for us if the rest is available. We have so much knowledge here that is hard to sift through. What does it take to develop this?
There are various approaches to doing this. Here is a summary of 2 techniques:
1. Retrain a small LLM with domain specific knowledge to generate a domain expert LLM
This is the one I am most interested in - but it would require lots of work prepared the training data from the forum content. And GPU resources to effectively train your own model with this domain specific prepared knowledge, so it's expensive to train even the small LLM's would require significant processing on Azure / AWS. And once it's trained it can only answer the questions on the data up to its training date (unless the other approaches are also applied see 2. below).
The advantage of this is it will be fast, relatively cheap to run, won't pull in content outside the forum.
The disadvantages are that its knowledge will be limited to how good the prepared data is, so will hallucinate if something is asked beyond its training data. With small model it is also like to hallucinate if asked to expand on some response.
There are examples of this type of small domain focused pre-trained LLM's appearing and I think they are very good (can compete with OpenAI / CoPilot) on domain specific knowledge.
2. Using RAG (Retrieval-augmented generation) with existing LLM.
The existing LLM can be OpenAI / CoPilot (effectively this is what Microsoft are doing with Co Pilot on your own content) or a small LLM. In an ideal world it would be output of 1. above but then this RAG algorithm would suppliment the trained data with up to date content.
This is the easiest to do in the short term, but can give good results but has problems. You effectively 'pre-index' relevant content from the forum (maybe just your FAQ pages when they are written might be ideal) in a graph database and before finally submitting content to the LLM (again doesn't have to be chatGPT / CoPilot) you retrieve these relevant paragraphs / forum posts) and used them as suggested content in the generated output.
The advantages are there are lots of open source tools that can do this relatively quickly and it becoming a well known technique.
The disadvantages are it is expensive (you get charged per token submitted into these cloud LLM like chatGPT as well as charged for content generated). So for example, if DAC is mentioned within a question, RAG would pull the relevant prioritised forum posts for DAC and submitted those with the rest of the question that was asked), so might be 10,000 tokens added to the question and this would happen each time someone mentioned DAC in a question. If you use this with a local small LLM you won't get charged but it slows down the generation of the response greatly (might end up being 30 seconds before it starts to generate a response.
The other disadvantage is especially if you use ChatGPT / CoPilot / Gemini is it won't limit the response to ASR content and will pull content from outside the forum.
Anyway, I can PM you with more technical details and examples.
