FREE: Verba Ollama GUI RAG Agent!😱 A.I. RAG By Weaviate (Open Source)!🤖🚀 Ollama Support - Llama 3.1B

What's going on everybody Josh pook here and in today's video we are going to be looking at the golden rag tever you heard that right this is verba by we8 and this is a rag retrieval augmented generating um op Source tool that you can self host easily I'm going to show you step by step how to do this and you can use this with models like GPT 40 as well as models like o llama local models llama 3 .1 whatever the

Case may be quen you know any single open model um coh here Google whatever the case may be we're going to show you exactly how to do that and you can upload files use rag in a nice uh easy to ous guey let's dive right into [Music] it all right guys so Burba is a tool I came across and it is a very interesting tool the team first of all like um you know still fairly new um I'm still fairly new

To it but just from you know using it a bit and and checking it out it's scen behind it um you can check out their YouTube channel they have like a bunch of videos a bunch of like how-tos Etc I mean it's it's really just the team behind uh weia we8 so they're a big company but um this right here I'll leave a link all links will be in the description as always but this is vera. weate doio so this is

Um essentially their their uh hosted s uh Cloud version of verba right so you can go ahead and test it out like this is basically trained on the verba doc so you can see here welcome to we8 your AI I guess it's trained on the uh we8 doc so um your AI native Vector database how can I help you with weate today so I'm like how do you how do I self-host verba and it basically gives me the um stuff here

You can see right here retrieve three trunks chunks in about uh 0.51 seconds and then blog we got this chunk this Chunk from installation and this you can see where it's from where the context is from you can even click on C context and see exactly what we're pulling from the rag all if you're not familiar with retrieval augment and generation done many videos on it just quickly summarize it's essentially um a method that's very efficient to use for llms it's

A very cost effective um method you know opposed from things like graph rag or more other more complex um in like setups but something where you can um easily query a system a database a vector stores and basically what they use like embedding models so an embedding model essentially takes chunks of text so say if I uploaded a large document it would chunk it into maybe let's say like you know x amount of chunks of x amount of you know uh

With x amount of uh text in there and then it would use um mathematical equations to essentially um store those chunks and those those words into um like basically numbers um into a uh into dimensions and then when the user query it it it does that same thing to the query so it changes the query into those uh numbers and then basically it uses mathematical equations such as cosine similarity Etc to um determine what is the best kind of related chunk

To that query and then it will be pulling those chunks and then uh it will get that basically allows you to have much more context within asking your queries to llms opposed from just having to you know put a whole bunch of context right we don't have to upload all of we v8's documentations to an llm and use like a million tokens worth of contacts we can just input it into a vector store and then query that uh database right so

That's just a quick summary uh of rag now we've covered tools if you're on this channel we've cover tools like rag flow and um a lot of people were liking this Jewel right I've definitely check recommend checking this video out as well this is a great tool for rag you can see a bit of the architecture right here showing kind of how rag works as well chunking um keyword embedding Etc so kind of just what I explained here's raglow too raglow

Is really good for multiple different um like unstructured data type stuff uh any sort of format but uh verba is actually really good for all that stuff too so here's verba just going to quickly run through uh some of the features and then we're going to install it I'd recommend if you're actually um you know really looking to get good at this check out some of these videos by the team here as well anyway so golden ragt all right so the

Golden retriever an open source designed to offer end to- end streamlined and userfriendly interface for retrieval augmented generation out of the box in just a few easy steps explore your data set and extract insights with ease either locally with hugging face and oama or through llm providers such as open aai coh here and Google so there's three main ways you can really install it you can do pip install golden verba right um and there's some other ways we'll get into in

Just a second so not going to cover every read this whole thing but essentially what is verbo fully customizable personal assistant for quering and interacting with your data either locally or deployed via Cloud resolve questions with your documents cross reference multiple data points or gather uh insights from existing knowledge bases verba combines state-ofthe-art rag techniques with we weat context aware database choose between different rag Frameworks data types chunking and retrieval techniques and llm provider on your based on your individual use

Case and then open source so features we got olama uh hugging face coh here Google open AI embedding support So support with a llama mini embed all these other ones right here different embedders um data uh data support So PDF ingestion GitHub gitlab so import files from GitHub and gitlab CSV XLS uh injection dockx uh injection multimodal this is planned unstructured IO so you can actually use this right here unstructured IO I'll leave a link to this too not going to

Get into uh using this but um this is another tool that you can use and you would just use that in your environment variables if you were to use that um and then rag features so hybrid search semantic search auto autocomplete suggestion and then a few plan ones is filtering Advanced querying reranking rag evaluation customizable metadata uh cool bonuses Docker support So This is actually how we're going to install it just cuz Docker just makes things easy I know it can

Take a lot of uh um computer memory and whatnot running a bunch of containers but just going to show you that for ease of uh just installing it you could also install this on something for example like coolify right you could easily just put the uh you know either forkit put your environment variables in or or whatever the case may be or just do a Docker file and then whatever you know you can just put it on duy or whatever you

Know server whatever you want to use rag libraries plan so Hast stack llama index Lang Ching so that's actually really interesting I'm glad that they're actively planning things and updating it so install via pip so you can do pip install golden verba build from Source get clone is another way so clone it and then pip install e and then use Docker for the M so if if you're not using Docker ensure you have python installing your system you're unfamiliar with python

And virtual environments please provide python read Python tutorial guidelines it's justest is to use in a virtual environment um and then API keys so via here you're going to want to create an environment variable oh there's an example one um where's the exam oh and the golden verba directory okay so I didn't even see that but okay so I'll show you that in just a second and here's the all the different variables that you can use you don't need to use

All of them but uh we'll get into that in just a second and then yeah that's pretty much it there's some more info here you can go ahead and check it out link for this will be in the description down below um but yeah let's go ahead and install this so what we're going to do uh just keeping simple is I just basically get cloned you can either get clone or download this uh and then extract the zip but you could

Just get clone um let me just show you as an example um what I did let's see so get clone okay so you want to get clone and then https GitHub weate for/ verba all right so make sure you got that going to clone as you can see um now if you did put the API keys in it's going to actually so first off what I would do is once you download I opened up the folder and VSS code you could

You could use vs code or you could use like just you could just open the folder text editor whatever you know you really need um so as you can see this is the environment example so you would change this I actually made one where is it I'm not going to click it because it has some uh API keys in there but I actually made one right here um is called EnV so and I just paste it in my own environment variables

But you can actually just use this uh example one and just rename it to say EnV so here you would put the um URL to your V WEA cluster and your API key cluster um so I didn't put this just because I'm just running it locally right now so it's just running off the the port um and then your open AI API key boom you put that in there your base URL um so yeah and then Al my URL um and

Then so GitHub so like there's all these other ones you really don't the main ones here I'll tell you which ones I used so you know this is how I currently set up mine I'm just removing the API keys and there boom so I didn't perfectly set it up like I only set up a few and I just removed the API Keys here but um as you can see I got my open AI I put a bit I I did go

Here even though don't even really use it um and then o so the olama um I have olama and bed model and then I have nomic in bed text latest and then my olama model is llama 3.1 latest and then this is where you would put your unstructured API key if you're using that and you get Hub token um anyways so uh yeah we're not really using all that and I kind of have some filled in that I'm not even using

So obviously it's not a perfect setup right now but um also too okay so just quickly too if you don't know about olama just quickly show you go to ama.com you're going to download it and then you go to models and you can really select any model you want it's very easy to um download them it's loading a bit slow right here but essentially would just select a model that you want and then in the top right it will say uh

Like AMA it'll show you the command you copy it and run it here we go so say I wanted llama 3.1 here we go here's how I install it once you install a llama then you just copy this and run this um yeah you can set the what what parameter model you want 8 billion 70 billion 405 billion obviously you're going to probably use the 8 billion if you're hosting it locally and um yeah so once you do that you can

Actually um do Docker compose up- d so make sure you're cded into the verber directory of course first so you're going to want to save your environment variables so you have that and then Docker compose up and then you'll see that it the pool was complete you'll see it tells tells me I'm missing some environment variables one thing too actually uh you may want to go into your Docker file your Docker compose here and if you need to change anything you

Could do so here as you can see um since we're hosting on Docker it's actually putting the AMA URL to this which you want and then these are the variables um what I did is I changed my port right here because it defaults to 3,000 so that's actually being taken right now one of my nextjs apps so you can change that if needed okay and then yeah you can change around things if needed but that's all I that's the only thing

I did just uh custom customizing the docker compose I just changed the support okay once you do that you're actually going to get basically this right so this is how it looks this is the self-hosted version first off I want to say it looks pretty cool I like it um I'm going to go over everything here right now so first off uh settings so so we have chat settings so we can enable caching Max character account to show documents enable auto

Completion enable help information okay title so I can actually change like I could say ride Josh try Josh and I could say [Music] Josh sub subscribe if you're still watching yeah if you're still watching and you're not subscribed definitely subscribe if you're getting some value here we upload videos every single day on AI automation business growth Etc and we just recently passed like two days ago 4,000 subscribers so thanks for all the new support really love of um you know seeing

All your messages and and support and whatnot helping you guys out anyways so intro message input message you can change that can add a logo if you want right let's say I wanted to add my company logo just grab that right here boom and there we have it I can even change not going to find like my exact color scheme right now but I could even like put the right right colors let's just say I wanted uh this right here I

Could go boom primary color Company color uh secondary color we'll just do like a little bit of a darker one and then accent color sure background color sure actually I kind of like I don't know if it will ruin it let's just try that oh there's a bunch okay so you can change the text color so wow alternative text color background color console so we can make like a dark like a dark uh dark mode I think anyways I'm not going

To so definitely not perfect like I said you could play around with it make it better right as you can see it's pretty cool already imagine if I was hosting this on kify on a custom on a custom domain um it looks like it's my own app right anyway okay I finally got it back so anyways what we can do is the I'm just going to go over the tabs so we got settings what we covered we got rag okay so

This is where you can select your embedding model right I'm using the olama one but you can use coher or ad EMB better all right as you can see right here and then select retriever we're using window retriever so retrieve relevant chunks their surrounding context using semantic and keyword search Hybrid and then select generator so you could use GPT 40 gpt3 command r or olama we're using AMA so right now we're using llama 3.1 meta's newest model okay then I'm just

Going to save next we can go to documents so documents we can basically select a reader basic reader GitHub reader unstructured API we're using basic so boom Imports plane PDF markdown Jon and docs files and then select a label for your documents for filtering so you could put document or whatever and then select a chunker so token chunker chunks documents by word tokens choose between chunk size and their overlaps you can choose chunks right here and your overlaps and then uh

EMB better so yeah same thing right so you can add a file here I actually added one already as you can see I just added a VSS code um basically just a document that gives like some vs codee extensions and best practices or what not now if I go to chat you can see here we're using our AMA and better um here reset conversation so this is the welcome message which we could have changed in settings and let me just say

Give me a step by step indepth the es code best practices and set up with extensions and tools from A to Z A2 Z there we go can see we're using the AMA batter and boom retrieving chunks so we got four chunks retrieved in 2.6 3 seconds all right all from the same document we could go ahead and see context if we want see all the different chunks boom me just cancel that I can see here the document what it's pulling

Now it's this a bit slow it's running locally but we can see based on the provided context I'll guide you through a comprehensive stepbystep process for setting up visual studio code with best practices and popular extensions please know that some steps might require additional documentation or resources which I'll mention when necessary so first step install vs code set up a repositories folder it's a little bit slow guys but um all right so we can see um you can divide in into

Personal repositories work repositories for better organization make directory give us some code here uh open a new folder in vs code showing us to shortcut code Dot um yeah so I'm not going to read through the whole thing but you can see here gave us some extensions git LS prettier e es lint and yeah you know it's still going it's a little bit slower but you can copy it here and uh all in all it's pretty cool it's pretty useful um

You can see that we still have our logo our branding and um I mean I guess that you can't I don't think you can remove this thing right here built with we8 so I guess you couldn't fully like white label or customize it but um other than that it's pretty I mean I guess you could probably just uh check the source code and if you wanted to but other than that guys it's pretty cool um pretty interesting um yeah I'm interested

To see if you know if they they say they have a bunch of stuff coming bunch of stuff planned in the work so once those updates roll out uh I'll be interested to continue testing it out but this is a no no code well very little code you don't really have to do anything it's a easy to use gooey interface for your rag system so may be wondering if you watch my other video on rag flow which one do I prefer

Well I think uh out of box just from seeing it I say like I I think rag flow is pretty um pretty robust and it has I almost think it has a little bit more features and a little bit more capabilities and you know it has that nice user flow with agents and everything um where you can like design it and stuff it's not perfect still it's not perfect um but this is pretty cool too um the the I like how

It has this us I like this user interface I like how you can see like the overview you can see your variables right here so you can add different variables you can see libraries admin console um and then you can add documents using different things uh different setups so and then you can chat with it so um and then yeah the white label and things that are kind of coming in the future and also too I like how the team has

A bunch of videos on this obviously it's made by we8 which has other products but um yeah I think it's uh definitely something to look out for and uh you can use it and let me know what your thoughts are if you have any questions or issues installing it or whatever the case may be let me know in the comments down below let me know what your thoughts are about this tool you have any tools you want me to do a

Video on or any videos in general marketing business AI Etc let me know in the comments Down Below guys like I said we just reached uh surpassed 4,000 subscribers so thanks for that um if you're new here definitely hit that subscribe button if you got some value here just stay up to date with the daily uploads also too if you haven't already joined our free community strid community.com our free Facebook group free Discord Channel link for that will be in the

Description down below along with all the resources I went over in this video other than that guys um also too if you haven't checked out my book client acquisition code link for that will will be in the description but I will see you guys in tomorrow's video keep hustling keep grinding and of course guys accelerate your stride take care