FREE: GraphRAG AI Is INSANE!🧠🤖 Microsoft Knowledge Graph RAG Better Context Than Vetor Store RAG!

What's going on everybody Josh pook here and in today's video I am going to show you how you can set up graph rag on your desktop system all right we're going to show you how to set this uh graph rag system up I'm going to go over also why graph rag is different than normal rag Vector storage rag or whatever the case may be if you're familiar with rag we've done a lot of rag videos on this channel but what is

Graph rag if you're new to graph rag we'll cover that answer those questions why you may use graph rag what are the strengths of graph rag over normal Rag and what are the weaknesses right we're going to dive into it show you how to use it let's Dive Right [Music] In all right guys so Microsoft released graph rag on July 2nd of this year so I'm not all the links that I go over like always will be in the description down

Below and I'll leave the commands too so you guys can easily just follow along um and make sure you guys get everything that I'm going to show you in this video implemented but um this is a research P or research blog post by Microsoft graph frag new tool for complex data Discovery now on GitHub all right so it kind of goes into you know advantages of community summaries for Global questions local questions what's the difference um going over different responses and

And all that good stuff I'm not going to go over that but I'll leave it in the description down below and they also have a great page right here on GitHub microsoft. github.io graph rag where it goes over getting started with graph rag all right installing it running it quering graph rag uh developing indexing architecture data flow all thing pretty much anything you could really think of when it comes to graph rag they got it here so I definitely recommend as

Well as prompts fine tuning prompt tuning I guess and a bunch of different so I definitely recommend if you're really trying to go in depth on graph rag to check this out and if you're new to graph Rag and maybe you're new to rag or you're new to whatever this is definitely the next step in in terms of like really um you know getting better at rag getting better with AI learning more about different um ways to query data and do

Rag systems so um yeah I'm excited that you're here and learning this either way we're going to show you how to set it up also too I'll leave a link down below to the AR iive uh uh research paper and it's this goes in depth explaining like graph Rag and you know Source document to text Chunk to element instances element summaries indexing time Community detection then graph communities Community summaries and then Community answers and then Global answers all right so pipeline

Stage chunk size has different graphs I'm just quickly SP uh skimming through some of this just to see if anything sticks out here but like I said I'll leave it down below it's pretty in depth when it comes to graft frag and it's definitely a good read to take a look at some of it okay and then also too of course I'll leave a link directly to the GitHub of uh Microsoft graph rag right here where you can check it out

Now you may be wondering like what is graph rag well actually this is actually a good question it I didn't even expect to have it right here but I'll quickly read some of the FAQs maybe this will uh help so graph rag is an AI based content interpretation and search capability using llms it parses data to create Knowledge Graph and answer user questions about user provided uh private data set all right what can graph rag do um graph rag is able

To connect information across large volumes of information and use these connections to answer questions that are difficult or impossible to answer using keyword and vector-based search mechanisms building on the previous question provides Semite technical highlevel information on how the system offers functionality for various uses this lets a system using graph rag to uh uh to answer questions where the answer span uh answer span many documents as well as a themantic uh question thema as thematic questions such as where are the

Top themes in this data set right so if you have so much data on all these different topics there's going to be themes and you know like communities what they say and uh that's we're we'll get into kind of more of that and you'll you'll get a better picture if you're new to graph rag but um it essentially allows you to query it in a much better way for specific data types like that right where semantic search it's really just um

It's just looking trying to pinpoint um you know a specific semantic uh relevance or similarity based on your query you know it won't be able to detect like bigger uh overarching Global themes or communities uh within your whole data set all right so that's what graph rag is kind of uh meant for so what are graph frags intended use cases um okay like there's some good questions here I'm not going to read through them all I'll quickly read through some of

These so graph frag is intended to support critical information Discovery and Analysis use cases where the information required to arrive at at a useful Insight spans many documents is noisy is mixed with Miss and or disinformation or when the question uses users aim to answer uh are more abstract or thematic uh thematic uh then the underlying data can directly answer graph regag is designed to be used in settings where users are already trained on responsible analytic approaches and critical reasoning is

Expected raag is capable of providing High degrees of in Insight or on complex information topics however human analysis by domain Experts of the answer is needed in order to verify uh and augment graph frags generated responses and lastly graph frag is intended to be deployed and use with a domain specific Corpus of text Data uh graph rag itself does not collect user anyways blah blah blah I mean do we know do we really I guess yeah it's open source so anyways

Uh made by Microsoft but let's quickly just look at some diagrams here just to really shoot some of these points home so um Regular Vector rag right here and then graph rag so just some kind of a gra graph here so data structure this is knowledge graphs if you're familiar with knowledge graphs you know if you've used like um what's that note ticking app um uh you know which one I'm talking a lot of you probably know which what I'm talking

About where it's shows like knowledge graphs Etc but yeah you it uses knowledge graphs Vector rag uses Vector database um focus so this focuses on graph rag focuses on entities relationships and context Vector uh rag uh focuses on on semantic similarity thematic relevance and retrieval speed so graph rag might slower for complex queries faster for larger data sets with Vector uh rag graph rag has few more steps especially if you're using Global search query query and scalability can be less scalable

For very large uh knowledge graphs and then Vector rag highly scalable for unstructured data deep understanding provides deeper insights through entity connections and then limited understanding of relationships and then the last image I got here is right here so uh by providing the model with with an access to vast repository of factual knowledge using graph uh the model can draw on know this knowledge to generate responses that are more detailed and accurate So Sophisticated searches uh graph R advantages scalability informative

Responses accurate query understanding explainability and then relationships and content context all right I'm just going to go over one more thing before I show you to get set up on it and I will include um time stamps through this video so if you already know about graph frag and you just want to show you like show know how to install it Etc use it you can maybe skip through or you can just watch we're about to get into it in just

A second but I did uh as perplexity just a few different things and I want to go over some of them just to give you a little bit more understanding before we get into it so I just said please explain the difference between graph Rag and regular rag not going to read this all but um so few similarities uh graph Rag and normal rag aim to okay so we kind of should know about that the difference is we kind of went

Over some of those in the last graph but here's some more so data structure uh Vector databases versus knowledge graphs information retrieval based on S similarity for regular rag of text chunks and then graph rag leverages structured relationships and graphs and then contextual understanding so limited to top K similar chunks in um Regular Rag and then graph rag can Aggregate and summarize information across a data set complex cury handling for regular R struggles with multi-op reasoning and then graag excels in

That and then yeah so pros and cons of regular rag Simplicity efficiency I'll also say yeah oh it says cost right here so cost it's way less expensive especially graph rag depending on what model you're using of course you can use it with llama AMA you can use it with uh open AI I'm going to show you how to use it with Gro that's another thing I'm going to show you how to use it with grock today just because um yeah

I don't want to be spending like a bunch of money on my open AI just for you know using graag I'd rather use Gro which is U free with obviously uh limited um you know limited uh queries and whatnot um and then exp uh explain explainability and then graph rag Pros context depth complex query handling um and then cons complexity and cost and then I just wanted to quickly go over two forms of querying SL searching in graph rag so we

Got local search as well as Global search so uh local search and graph rag focuses on specific entities within the knowledge graph it retrieves information by Fanning out to the neighbors and Associated concepts of the query entity and then use cases ideal for queries about specific entities such as names of people places or organizations and suitable for detailed entity specific information retrieval so Pros it's Precision so provides highly relevant information about specific entities efficiency so faster retrieval for enti uh specific

Queries and then con so scope limited to immediate neighbor of the uh queried entity which might miss broader context and then we have Global search okay where you're um going based off communities and whatnot so Global search in graph rag leverages the hierarchical uh structure of Knowledge Graph to reason about holistic questions related to the D data set it uses Community summaries to provide comprehensive answers so that's what it really so basically it uh leverages those Community summaries so you're going

To get a more holistic uh View and then use cases so suitable for broad them thematic queries that require aggregation of information across a data set ideal for summarizing main themes or Trends in large data sets Pros comprehensiveness so provides detailed holistic answers contextuality and then complexity is for the cons and latency and then of course PR uh actually no the prices that's for graph rag in general and then just lastly I'm just going to go through the the exact process

Kind of step by step of graph Rag and then we're going to get right into it so not I'm not going to read through this whole thing but um so pre-processing and indexing so text Unit segmentation the input Corpus is divided into smaller text units chunks to facilitate detailed analysis these units can be paragraph sentences or other logical segments so a lot of graph uh graph rag like the step by step is somewhat similar to um to regular Rag and then

There's that it's mainly the communities there we go entity and relationship extraction so an external large um language model like gp4 for example is used to identify and extract entities people places organizations and their relationship from each text unit this steps these steps involve entity extraction so identifying key entities within the text relationship extraction determining the relationships between these entities and then the extracted entities and relationships are represented as nodes and edges is in a knowledge ra all right hierarchical clustering

So using algorithms like Leen uh technique the initial Knowledge Graph is clustered into communities these communities represent groups of entities uh that are densely connected and then Community summarization uh summary gener uh generation so summaries are generated for each Community capturing the main entities um their relationships and key claims this provides an overview of the datet and useful contextual information for subsequent queries and then retrieval and answer generation so query mapping incoming queries are mapped to the most relevant context within

The knowledge graph this is done uh iter iteratively uh to ensure the best match and then context augmentation response Generation all right I'm not going to read through the whole thing it's kind of similar and um yeah so just wanted to go over that I know it's may have took a little bit time but I know some of you are new to graph rag so I really wanted to give you a bit of a little bit of a lesson there just

So you are on the same page now now that we're going to actually um set up graph rag on our system so uh I'm just going to be using like uh this Romeo and Juliet txt I just got from gutenberg.org this is the thing that we're going to be embedding into our graph rag of course but you can put your own data in of course all right guys so now we're going to install graph rag so essentially what you're going to

Do is you're really just going to run this command pip install graph rag you can do this in your terminal or in vs code you may want have to run it as administrator just to have necessary permissions but you're going to install it and once you do that depending on if you're on Windows or uh Mac or uh Linux you're if you're on Linux you can do export graph rag API key set the environment variable um but if you're on Windows

Like I am right now you're going to do set X graph rag _ API key uh UND like this and then quotations and then your API key and this will be our grock key okay like I said you can use any you could like if you're just using open AI then you would put your API key here for open AI but um since I'm going to be using grock for this I'm going to put my grock API key here all right

Then um I'm also going to put my open AI API key right here because we're going to grock doesn't have embedding so we're going to be using open AI for embedding okay now um what you're going to want to have in VSS code or whatever IDE you're using is you're going to want to open up a new folder so I just created one for the sake of this video called graph Rag and um all I did was I put created a

New folder called input right here and then I added a.txt file just called Romeo and Juliet and I just as you can see here I pasted in my text it's about 5,600 lines of text and it's written I mean it's new line it's small text uh like it's not the full entire line it's because it's like Romeo and Juliet but anyways um once you do that you and like I said these prompts are going to be in the description down below

So you can literally just copy and paste and follow along you're going to run python so you need python installed that kind of goes without saying but just make sure you have python you're going to run python DM graph rag. index and then D- init to initialize and then D- root and then dot space dot right and this basically just means the current folder you're in so you're going to want to run this in that folder all right I actually just

Did that as you can see here and it's going to start initializing okay so once you do that you'll see that like you got these um these new folders in here so you got like the prompts you got your environment Vari environment here so you could actually just add your environment variables here but if you already exported them as environment variables in your system then you don't need to do that okay but you're going to want to go to your settings

Reason being is because we have to do some configurations and you can do different configurations I'm going to show you how to configure it just the way I did but you can do it differently as well as like you can there's also a lot more stuff you can do here you can see that there's different uh things here so if you want C things here you could like remove them like remove the hashtag right here but anyways um so first off

You can see here we got API key grab frag type so as since I'm using Gro I got mistl X or seven or sorry 8 x 7B 32768 just because that's the higher context um model on grock and then uh the next thing is the API base so if you're using grock uh like me you put um https api. gro.com sloping ey slv1 but you can also use a llama with this um so yeah you would just put your API base

4 o llama or or whatever the case is if you're using open AI you don't have to even do this you could actually just keep it the same and then tokens per minute I put it to 2,800 requests per minute I put it to 28 Max retries I put it to three Max uh retry weight I put it to 13 and um yeah graph rag like I said before it can like if you're using like open AI or something it it's

Going to use a lot of tokens just keep that in mind so that's why we're we did some of these settings and then embeddings so we're going to want to that's why so you're going to want to change if you're using um grock and grock for uh the tokens and then open AI for the embeddings you're going to want to change it it would have said graph rag you're going to want to change this to open AI API key and that's

Why we exported it exported both of those keys or set both of those keys all right so you can see here we got the text embedding three small model um and then uh let's just see everything else I pretty much left the same I did put for summarization um no I did put for let's just see Community yeah it's Community I change this max length to 1,000 and Max input length to 3,000 but you can see here we got a bunch

Of different options that you can potentially use or play around with you know what you're doing or just learn about it uh you can ask chat GPT and kind of learn about these different things but cluster graph and bed graph um map snapshots and I'll be doing more videos on graph Rag and different uh variations of graph rag as well so make sure to stay tuned subscribe if you're new to the channel we upload videos every single day on AI automation

Business growth Etc um anyways once you do that we're going to go to our next command so instead of running graph rag index in it rout we're going to run graph rag index root dot so without the init okay so we already in uh initialized it now we're going to actually index it at least the root location which is here so we're running that now graph Rag and I actually did encounter a couple issues so I did make a few changes

I'll just let you guys know and if you do encounter any issues um yeah it's you know there's a lot of different configurations you can do so you're going to have to maybe you run into any issues you may have to troubleshoot um but I'll I'll go over what I did in just a second but here I just want to show you it running so we can see here I went through a bunch of different steps so create base unit uh

Base text units ID chunk ID chunk uh different chunk Etc um entity gra so uh create base extracted entities create summarized entities create graph entity uh create base entity graph create final entities so we can see different entities the different names here you know type so person Geo um you know graph embedding text Unit IDs description embedding all right can see different rows times columns and then create final nodes so we can see like title person description is the prince of

Verona Source ID graph embedding entity type top level note ID um create final communities so we have the different communities the relationship IDs uh text Unit IDs join text units to entity IDs uh create final relationships so we got that uh join text units to relationship IDs create final Community reports create final text units and then create base documents create final documents Etc and you can see everything all workflows completed successfully so there you have it um then you would have

Within your output you would have the run that you use the artifacts and you can see all the different artifacts create based documents all these different ones that we looked at you can see the stats the reports logs Etc so if you do encounter any issues check your reports and your logs your index engine logs too sometimes um and basically what I did is so a few I was ining a couple different errors the main main thing I just honestly just

Decided I just switched it out back to open AI API key so you could keep graph Rag and just uh put that as your open AI key and just only use that but just CU I already uh set my open AI key I just change it to that I just Chang it to GPT 40 just for Simplicity and then I just uh marked out these options because we didn't I didn't really need them um because just based on like the content

The amount of content I was giving as well as the settings I had I kept running into I think rate uh rate limits with Brock um you can try it though with grock you may or may not encounter the same issues so if you don't let me know but it could be based on the plan you're on or Etc so I just personally was so I just changed it to open AI for Simplicity sake and um yeah everything else I mean

You can you can mess with the chunk size if needed overlap um but but I the main thing I did was just change it um to open Ai and yeah also too guys if you want me to do future videos showing like uh how to use this with olama and local models let me know in the comments down below I I definitely will be looking into doing that just because guys like if you're going to be actually using this on a

Consistent basis it will like I said get pricey with open Ai and like paying for it cuz it it is a lot more labor intensive than a normal r system okay but anyways now that we got that out of the way let's go ahead and do some queries so I'm going to show you two query methods um the first is doing the global method now this is going to go based on the community summaries um include that and then the local

Method so like I said at the start these two methods um have different kind of strengths and you know depending on what query you want to do so for here for the global since we're going to be referencing the community summaries we're going to ask what theme is in the so to do that you do python DMC rag. query D root Dot and then D- method Global and then in uh quotations what your query is going to run that all right

Guys so one thing I would just want to point out is I actually did encounter just a quick error just because I forgot the Double Dash for rout right so it does require two dashes so just to show you again and this will be in the description down below but uh make sure you use two dashes like so for for the route um obviously I just messed that up but anyways as you can see here check the settings and reading settings.

Yo and then success Global Response main themes in book so the Rivalry between the Capulet and the mono families and then the rule of the prince in maintaining order impact of the feud of rona's citizens and then influence of key family members loyalty and aggression among servants and then conclusion all right so pretty good this is a lot better and a lot more uh Inc like you know able to like normal graph or normal rag wouldn't be able to really um

Pick up on that whole Grand picture of things when you're querying it so that is what we did with the global uh method now let's do a local method so this will be based on um a more pinpointed question such as who is Romeo and what are his relationships all right so boom we got our local search response so r and his relationships Romeo's identity Romeo is a central character in the community of uh community of Verona uh known for his

Pivotal role in the unfolding drama he is the son of monu who one of the heads of the feuding families in Verona um and you even see references here right uh the family of um and yeah so key relationships uh Benvolio Benvolio Mero uh mer Q um yeah so you can see the relationships here Bazar and all that good stuff I I haven't read Romeo since high school but uh awesome so yeah there you have it guys I showed you how

To do a local query a global query we went over what graph rag is why it's different than normal rag as well as how to actually set it up like I said there's a bunch of different ways you can configure it we can set up a local models you can set it up many different ways and you can use it in your applications and uh yeah this is really just the start um but and there also is other different rag ways

Of doing things that um I will be covering uh in future videos so yeah I definitely think that this is just kind of a stepping stone this is not the end all be all there's still a lot of pros and cons to graph rag that I think you know in due time uh some systems already do fix them or mitigate like cost and whatnot but more and uh as just said with anything with AI as it continues progressing we're going to

See lot of these cons um not be there anymore right as things get cheaper more computer Etc so anyways guys if you're new to the channel we upload videos every day on AI automation growth uh business growth if you're entrepreneur AI Enthusiast developer coder business owner marketing agency coach consultant whatever the case may be we have almost 400 videos on this channel right now and we're uploading every day we're not going to stop so if you got some value here make

Sure to smash that like button smash that subscribe button really trying to grow things here got any value whatsoever I definitely appreciate it um and other than that guys if you have any questions or issues let me know in the comments down below you have any suggestions on future videos if you want me to cover a specific topic whatever the case may be let me know I want to engage with you guys as much as possible you know we're growing still

So I will be trying to see as like I probably will see most of the comments so there is a good shot I will actually make if it's a good idea a video on that topic so um and then also too guys if you haven't already joined our stride Community free group free Discord free Facebook group Link in the description down below um we have about 750 800 members uh you can network with myself as well as like-minded other entrepreneurs and

Then also too guys if you haven't checked out my book client acquisition code and you run a business it's going to show you how to pack your calendar full with hot warm qualified appointments through many different methods uh outbound prospecting paid ads Etc I have a full chapter on AI and prompt engineering it's like very in-depth it's no fluff so definitely check that out Link in the description Down Below guys other than that check out some of the other videos on

The channel subscribe and I will see you in tomorrow's video guys keep hustling keep grinding and of course guys accelerate your stride take care