Graph RAG is great, but can it eliminate hallucinations?

RAG has been a fundamental instrument in molding GenAI to work for the enterprise and the use cases that have been identified. Vector RAG is great at implementing faster and efficient similarity searches but at the cost of poor accuracy and reliability.  Vector has now been phased out by Graph RAG that is quickly gaining popularity for its benefits of storing data in a structured and traversable format. While this is a step in the right direction, it is not the answer to all the hallucination and reliability issues. At least not on its own.

There are a lot of frameworks available today that are great at pipelining, for getting out data in the right format and putting things together. These frameworks are not bad at what they do, it’s just that they don’t solve the entire problem. The industry currently lacks a complete solution that is reliable and standardized.

If you look at the initial stages of the pipeline that involve extraction and preprocessing of data, you’ll start to see why data is a crucial component of this process. The better you handle data, the better your chances of grounding the LLMs are. How you persist the data, the ability to interconnect relations and concepts and the ability to handle different formats is crucial.

If you look closely at the slide above, on the top you see what the typical pipeline looks like but with a focus on ensuring the language model is used for its strengths, i.e. understanding the data that has been plugged in. This means that you will have to handle cases differently based on whether it’s a conversation, a document, a video or a table. This is what CogniSwitch enables developers to do well.

At the same time, we also believe that it's crucial for domain experts to remain in the loop and help in the curation process. So, relying only on the language model to understand common language concepts and business domain concepts is a poor approach. Just as a business would onboard a new employee, it's crucial that you ensure that you shadow their activities and ensure things are being done the right way. This will go on to ensure that the knowledge is vetted by the business before it’s used in an application or for an agent.

At the same time, we also believe that it's crucial for domain experts to remain in the loop and help in the curation process. So, relying only on the language model to understand common language concepts and business domain concepts is a poor approach. Just as a business would onboard a new employee, it's crucial that you ensure that you shadow their activities and ensure things are being done the right way. This will go on to ensure that the knowledge is vetted by the business before it’s used in an application or for an agent.

To conclude, winning this game requires:

• Enterprise data to be stored and structured in a format that makes it easier for the LLMs to consume and reason with.  

• Neither Vectors nor Graphs on their own are going to give you the level of accuracy that is acceptable to push into deployment.  

• Fusion RAG mechanism is further enhanced when rules and decision engine, aka Symbolic is utilized, which is a forgotten but very useful technology that goes on to give you the ability to do reasoning and inferences.

‍

For a deeper understanding of these concepts and to experience the difference in the capabilities of a Bot powered by Vector RAG vs a Bot powered by Fusion RAG, I recommend watching the event we held with the folks at Tars. We get into the limitations of Vector RAG and how you could go about overcoming them with a Fusion RAG framework. Here’s a link to the event.