FREE: Open-Source RAG Kotaemon AI🤖 GraphRAG Chat w/ Docs Locally 📄 Retrieval Augmented Generation

What's going on guys Josh poock here and in today's video we're going to be looking at an open-source rag based tool for chatting with your documents now this is not just any open- Source rag based tool for chatting with your documents because we've covered multiple different ones on this channel we've covered verba we've covered lobe chat we've covered many different ones as you can see here as well as many other different open source tools if you're new to the channel we

Upload videos every day on these things but this one specifically is clean it's customizable for rag specific chat and it also has graph rag functionality as well as normal Rag and it is very um customizable let's just say so we're going to show you how to you can set it up within 5 minutes locally 100% for free let's dive right into [Music] it all right guys so this tool is called Kon uh I hope I'm saying that right but yeah you

Can check out the link Down Below in the description it's a GitHub uh repo so um yeah check that out as well as all the links I cover will be in the description down below now I'm not going to read through the whole GitHub here it goes over the different features like you can host your own rag um document QA web UI organized llm and embedding models hybrid rag pipeline multiple modal QA support Advanced citations and documentation preview uh support complex

Reason in methods configurable settings UI and extensible uh extensible so yeah a lot of different things you can use it with open source models like AMA as well as you can use it with uh you know models like open Ai and thropic really the list goes on so um this is what the user interface looks like and without Ado let's go ahead and set this up now this is actually one of the easier tools to set up it's very very simple

There's two main methods with Docker or without Docker so I'm going to show you the easy recommended method which is with Docker and so you're going to have to go to doer. install that if you don't have it already and you should have it because um it's a great tool Docker it's 100% free too so what you're going to do simply is if you're on Mac or Linux you should just be able to run this Command right here exactly as it

H uh as it shows within the GitHub um and make sure you have your Docker open while you run this command in your terminal now if you're on Windows you're going to want to run this command which is um looks something like this and this command will be directly Linked In the description down below so it's basically just Docker run um server name right there server port 7860 and then right here and then the um the docker Right image right here

Um so yeah it's pretty similar it's just customized for Windows so links for that or that will be copy and pastable right in the description down below check that that off your own Windows once you run that it's actually all done it's going to pull the image and it is going to launch it on Port e for 7860 okay so once you do that you're going to be able to uh access it and it's going to look like this so this

Is UI I'll quickly go over that um right now so we got the chat interface we got the files so this is where you can upload files uh filter by name oh wow this is cool I didn't even know this so they have like a database of your files that you uploaded now this is your file collection side of things and then this is the graph rag side of things all right if you're not familiar with graph rag I'll maybe link

A video down below I did on graph rag it's a bit longer and it's honestly quite to be quite honest um graph rag is it's not something you probably want to be using on day-to-day basis for it's very specific just due to the fact that uh it does cost it runs up a lot of tokens there's ways to get around that but uh yeah that's just a little side note here now resources this is where you're going to come to actually

Change your llms your index collections if you want to use grav rag as well as your embeddings and all that good stuff so within index collections you'll see it's by default just set to the normal file index and this is what I'm going to stick to because CFT rag yeah it's like I said it's a little bit more complex but if you want to do it you could use it and it's pretty uh useful for like um extracting entities and all

That I have a full video that's about 30 minutes long where I show you how to uh self set it up graph rag on your system and and actually do all that so you want to check that out learn more about graph rag look at that um and then llm so this is where you can um you could manually add different llms from different vendors right here so we got different ones right and or you can just go to view right

Here and you're going to be able to see the default um llms that are already added so we have grock we have Claude we have old llama and we have open AI now I'm not going to click on the open AI one this is that's currently my default one right now and that's where I have my API key um once you click on it it's going to you're just going to paste your API key right here base URL is going to

Be already there and then you're just going to click on Save and then you can even test it so that's really useful as you can see though too there's AMA there's Claud all these different ones so if I wanted to change it all I would do is just change this from True to false by clicking on it and it's useful tool because it even shows you different um different uh you know um variables and parameters that you can actually add to

Here so like if you're in the open aai chat model you may want to add like frequency penalty or um uh Max retries or Max tokens or whatever the case may be and it has a description there and it explains it all so really useful stuff right there now if we go to embeddings we can see a similar thing just for embeddings we see oama open AI local BNG and um yeah I'm using the just the default open AI one and

Yeah it's same s same sort of thing if you click on it you can change your API key you want to change things you can just toggle it to true or whatnot you may have to restart the app once you uh actually no I don't believe for this but uh if you turn on graph you do have to restart the app and then you're just going to click save and then test and you should be good to go here you can

Also see the different users now I should mention that the default user once you um log in or once you get this spun up is admin and admin so admin as a user admin as a password and you can go to your settings right here I believe is it settings um somewhere uh I we'll get to it in a second but you can oh here user settings you can change the password right here and then we have retrieval settings so file

Collection and graph rag you can customize settings here both so you can see different ones here okay depending on what you're using retrieval mode so hybrid text Vector store um graph rack and change settings here as well search type local or Global not going to cover like exactly what that means like uh I'm not going to explain graph rag like I said if you want to learn more about that Global enti or search um local search you know I talk about

It there um and then language English reasoning settings Max context L uh length reasoning specific settings complex react reasoning options all these different things so I'm yeah just there's some different options configurations there I'm just skimming over some of the these so you kind of get the gist and then there's a help section here and then also too guys there is their docks right here which I'll leave in the description down below so list covers pretty much everything if if you

Need a refresher on anything I went over here check out the docs and if you have any questions as well you can always ask me in the comments down below now let's go ahead and test this bad boy out okay so to do so we're going to go to chat and what we're going to want to do is upload a file okay so I have a file already and you you can do a doc you can do a PDF there is

A lot of different um you know uh formats you can use as I upload it you can see it's starting indexing of one file so it's beginning indexing and if everything goes well in just a second we should see index completed okay so now I can actually query or do a Q&A about that specific doc so this document is just a document I did a video on this document it's just about uh 10d uh a2p 10 DLC registration um for uh

Like registering a twio phone number or um whatever the case may be within go high level if you don't know what that is don't worry but I did cover that in a video um okay so what's the best practice for guaranteeing ATP approval make sure you select the specific file you're using you can either search in a specific file or files or search all and then we're going to click on send and if everything goes well it should reference and do

Rag retrieval augmented generation within this document to to answer the question so as you can see here we have the information panel on the right hand side where we can see the different pages and the context that is uh pulling from the specific uh for the specific question we can see a relevant score based on each page and we can see the different text it is parsing or looking through um so we got a relevant store 0.8 and you can look

More into this I'm not going to go over that but we can see we get an answer that is based on our specific document and best practice right here for a2p accurate business information campaign you uh campaign uses uh use case a sample message it gives the exact sample message embeddable link and phone embedded link and phone numbers uh user consent opted message acknowledges acknowledge fees and charges and compliance with fls so really cool stuff of course you can copy it

Like dislike and you can regen too so regenerate if you like and you're going to get a new response now here is the shared button to I believe mark this chat as a shared um you have other options here such as edit the chat name uh Delete the conversation make a new conversation and all that good stuff but yeah that is the gist of this tool um the reason I'm covering this tool is because one it's one of the better rag

File systems that we've seen that I personally seen out of the other tools that I've covered on this channel if you haven't seen some of the those other ones you may want to check them out glob chat uh different ones like this and I'll maybe leave a couple links down below if you want but this is also a repo that is trending this week as number one um on GitHub so uh it's a based on python mainly and yeah it's you

Know a lot of people are liking it it has over 8,216 Stars so I took a look at it um just because you know we've covered a lot of these tools on the channel and this is definitely um a really good one especially the graph rag functionality like that's pretty cool that they have that uh other than that guys that is it for this video if you're new to the channel we upload videos every single day on AI automation business growth

Open source tools AI coding whatever the case may be we're consistently putting out content when I'm learning things I'm going to share with you guys if you have any questions or suggestions on what type of content you guys want to see next I'm going to be making a lot more videos on ai ai coding cursor lot of different stuff go high level if you're not familiar with that I have a lot of videos on that I'm going to be making a

Lot more videos so I'm going to continue inre ining the quality and trying to improve every single day when I create these videos so feedback from you guys was appreciated um and yeah other than that guys if you got some value from this make sure to smash that like button smash that subscribe button to stay up to date other other than that guys I will see you in tomorrow's video keep hustling keep grinding and of course guys accelerate your stride take

Care