NEW: Complete n8n Vector Store RAG Setup Using Zep AI, Ollama, OpenAI (HuggingFace) Self-Hosted

What's going on everybody Josh pokok here and in today's video we are going to be setting up your very own rag system in n8n retrieval augmented generation and we're going to be using zap oama and open AI but you can use whatever models you want we're going to show you step by step how to set this up right now let's Dive Right In [Music] okay guys so I've done many videos on n8n I've done videos before on zap I've done videos

On olama but I'm just going to go through all those briefly show you how to set this up in nadn and so you can have a rag on your very own nadn instance and ideally I mean if you're self-hosting NM which I've covered in many videos so I'm not going to go through the whole overview of n8m um this is assuming you kind of know what NN is if you haven't check out some of my other videos but you can easily

Self host n essentially a or zapier alternative that's a lot better anyways so now that we got that out of the way um basically what you're going to do is you're going to have an self set up and if you haven't done all that you can check out my video I did yesterday as well and you may want to do that anyways because like I said you can use any model right you could literally use any model you could use hugging

Face you could just use open AI if you you know depending on what the case may be but in this specific case I'm going to show you how to to do it with AMA so if you want to have olama set up on your on your uh say if you're hosting your N9 instance on a VPS if you also want to be able to use olama models open source models on your n8n on your VPS and you're going to want to

Check out my video I made yesterday because I show you how to do exactly that anyways so this is olama um check that video out link will be in the description down below I'll also link a video I made a while ago about zap but I'm going to quickly cover it uh right now so Zep right here you can do a lot of cool things with Zep um I'll do more videos with it but um this is just really showing you

How to create a basic rag system and that you can iterate on and improve upon right so Zep long-term memory for AI assistance recall understand and extract ideas from chat histories power personalized AI experiences okay I'm not going to go through the whole entire repo here I'll leave a link to that and this is their website pretty cool uh tool you can integrate in your in your development in your apps in different Lang languages of course you can see here na

Uh n python typescript go Lang Chang Lang flow flow wise so all the top tools they have a Discord they have pricing if you are using their Cloud hosted one I'm going to show you how to self host it and their docs are right here and I'll leave a link to to the docs right here which literally walk you through self-hosting it it's very simple so um to actually get started with anen um or with zap you're going to go to

The quick start right here and you're literally going to run these commands get clone and then you're going to clone the repo all right I already have the repo so I'm not going to run it but that's what you would run then you're going to CD into that directory so you would go CD Zep so change directory into Zep okay so um just see so right now I'm in my Zep directory as you can see here like if I went like

This now I'm in repos so I would CD into to Zep after I cloned that and then what you would do is you would create a EnV file okay I actually already have this file created um but you would create thatv file all right just if you don't know how to create a if you don't know how to create a file on Linux just catch apt it create thatv file and you're going to want to put zap open aore API _

Key equals and then your open AI key at that point you could start the server but what I would suggest is to um put some of these so you would want to put this one right and I'll leave links to all this in the description so you can copy and paste this these are just the environment variables so Zep open AI key obviously put your open AI key right here um and then Zep server web enabled equals true and then this

One I don't even know if you necessarily need it I'll leave it just in case um but you may not even need to put this one to be honest I actually don't think you do I was just I put it there because I was doing something else but um I'll put it there just in case you want if you if you do need it and then Zep off required you could put that as true and then um Zep off secret so

We're going to get the secret in just a second and I'm going to show you how to do that so once you do that what you could do actually is you could so what I would do um once you put the those environment variables in um I would just spin it up real quick so start up the server uh Docker compo uh Docker pull compose pull and then Docker compose up and then this uh this will start the Zep server on

Port 8000 okay and uh I put mine on a different port just cuz I was using 8000 8000 is a pretty common uh port to use so you could just started up on a different um server uh and like honestly I would if if you don't know how to do anything like server commands or or whatnot that I'm like Linux commands that I'm going over I would literally just say how do I Docker actually what you could do um you could

I think you may be able to um I was going to say how can I Docker compose up on a different uh Port so you could probably do it that way and just get to chat GPT command or what you could do is you could go into the docker compose right here here and you could basically uh clone this repo or you could Fork this repo before um before get cloning it onto your server and edit the docker compos and just

Change the docker image right here instead of it saying 8,000 by 8,000 you would put 4,000 or 5,000 or whatever you want 3,000 any other Port by 8,000 okay that's just going to point the that so that's if you're already using Port 8,000 for something okay so that's one Surefire way you could do it I hope that makes sense you have any questions let me know once again put it if you need to if you already have 8,000 taken then you

Can just clone the repo or Fork the repo in your GitHub account and then um change that Port here and then get clone it into your server okay so change a port first if need be or you could just use port 8,000 if it's not big deal for you if you are using coolify though which we've talked about in many videos then uh coolify uses Port 8,000 so you're going to want to use a different port okay and then they have

A SDK if you're you know incorporating uh zap into like your apps or anything um production let me just go back so next Once you Docker compose up you can go to your server IP and that Port so say if it's Port 8,000 or Port 3000 you go to your port and you'll be able to access the zap uh interface which looks like this you can see dashboard join our Discord users sessions this is a new instance uh collections okay you

Can just see one right here settings okay and then all that good stuff okay so um what you're what we're going to do now is authentification so just to cover one more time what environment variables we're going to want to put in so you have your open API key you have the server web enabled and you have off required true um I believe you need that and then your Zep off secret so how do you get your Zep off secret well

There's two ways and like I said this will be linked in the description down below if you want to do it the other way because uh you can use the Zep CLI um I didn't do it this way I'm going to show you how to do it this way so we're just going to generate a secret so this is literally what you do you would just go um into a for example another you could do it really anywhere but I'm like

I'm not going to SSH into my server right now but I would SSH into my server I'm just in a new tab and then you would run op SSL Rand dbas 64 and then space 64 so this is going to uh get you a um a specific token so b or it's going to get you a um a line Uh u a hashed uh code right now once you get the hashed code it's going to look like something like going to

Look like I don't want to show you but it's going to look like something like this okay in your uh terminal so when you copy and paste it from your terminal you're going to have to remove the line break okay I just hope that makes sense so like let's say I had my hashed thing right here like this was a hash it's going to be on two lines it's going to be like this and like this what you're going to want

To do is you're going to want to copy it for Le past it over into a notepad and then you're going to want to remove that line break so it's just all on one line okay hope that makes sense now that you got that you're going to go over to JWT doio link for that will be in the description down below and what you're going to do is now that we have our hash uh token or code right here we're going

To like I'm going to refresh this you're going to delete this right here from the payload like that and you're going to go into where it says your 20 uh 256bit secret and we're going to paste this in okay once we do that we can copy this encoded um uh to uh token right here so that's what we would use for our Zep off secret okay so that's the Zep off secret that we would put which is this right here all

Right I'm just showing you a dummy one of course and then once you put that in your environment variables then you can um save that make sure you save those environment variables and before you obviously so yeah and then once you save that you're going to um you're going to Docker compose down okay so I'll show you that command and I'll leave like brief um descri uh instructions as well in the description down below so if you're are not you're having

Trouble following along you'll have text um step by step telling you what to do so once again Docker compos down once you once you change your environment variables so you could Nano Dov to go into your environment variables change those make sure you get that code in that we got from JWT and then Docker compose down and then Docker compose up- D to restart your zap server uh zap application okay once you do that you'll have everything set up with zap

From that point on what you would do is you could go to n8n if you go to home here there's a place where it says credentials you could click on credentials I'm just going to show you cuz if if you're you could do add credentials that way or you can also add credentials within the workflow Builder which I'm just going to show you how to do just because I'm already here so I would just search for Zep and you can see

I already have a Zep account right here let's just pretend I didn't have a Zep account what you would do is you would come here you would paste in that um the that secret that all secret would go right here for the API key and your API you URL would be your IP address and that Port all right so or like if you have it connected to a domain either one of the two let's just say for simple use you know

You're putting it your IP address HTTP colen slash and then the Zep URL right um right here this URL and don't put the forge slash oh yeah so I one thing I forgot to mention so this is actually important um when you if you're having trouble going to your Zep dashboard at the start um I didn't mention this when I first said this but you actually have to go to your IP address and then colon the port so let's say Port

3000 or whatever and then forward slash admin so I'll leave that in the link in the description too just so you guys got that for SL admin but when you're in N you don't want to put forward SL admin you just want to put your IP like HTTP this IP address colon Port 3000 that's it okay hope that makes sense I'm going to cancel that uh close cool it's I already have my Zep account now you should have your Zep account

Connected into n8m now from that point I'm going to walk you through typical this is like a very basic rag system in naden but it works and um yeah you can use naden first off I'll just quickly say you Zep isn't the only Vector store of course that you can use with an end like if I detach uh this right here oops o I didn't mean to do that UND do if I delete this line I click here there's multiple Vector

Stores we got inmemory Vector Store Pine Cone Vector store uh cudr Vector store superbase Vector store and zap Vector store so I've done a couple videos on V uh super base and zap really like those two Pine cone's good cant um I actually haven't really used too much of it but and then in memory stores Bas is basic if uh you're just keeping it within the memory it's really not um Advanced or anything zap is really good I really like zap

And I really like super base um and yeah I'll do more videos on different tools and whatnot in the future about naden anyways so what we're doing here is we're just starting very simple uh this I want you to keep in mind that input right here of the data and what you do with the data as well as on both ends um this is where like you would add your own customizations I'm just showing you test samples right here of that

The rag working but then of course you could integrate this into really any workflow like I could connect air table here I could connect my notion I could connect click click up I could connect go high level I could connect anything and I could set up a rag system with it right and also too with a rag you can even there's a lot of stuff you can do um that's a lot more advanced than this but anyway so we're starting with

A click uh action trigger and then I'm just setting do using a set node right here and this is just dummy text and I just paste it in a bunch of text from uh Romeo and Juliet by William Shakespeare I just pasted in some of this just for the sake of this video but of course like I said you could scrape a website past it in you could put stuff from uh um like something to regards to your client or a

Project you're working on whatever the case may be anyways next we're adding a zap Vector store note so click on you search Zep Vector store and when you there's you may see it's like not this uh size so you w click on it and you're going to want to change the operation mode make sure it's insert documents okay once you do that you can add an option already added the option here but I'm just adding an embedding Dimensions okay so with

What we're doing here like I said you could use olama you could use open AI you could use hugging face you could use really anything and different embedding models as well as different uh just models in general some of them have recommended or specific um Dimensions or chunking sizes uh sequence sizes that you want to look up and you may want to buyby or you know whatnot so for example like if I'm using this is nomic um this is the one

I have installed on oama but I'm just showing you on hugging face if I scroll all the way down um you can see we got the sequence length here uh Dimensions right here 768 for this one uh there's different like you know ones right here you can also check on on the mte leaderboard if you're like using hugging face models for embedding um if you're using opening eye for embedding there's also they have a full section on embedding and their help

Center so you can check that out if you're curious about like if you're using a specific embedding model and you don't know what dimension size or um chunk size you should use just as chat GPT or perplexity um to give you a rough idea or you if you really don't know and you can't find it like I want I would Google it but if you don't really know don't find it you could copy some of the settings I'm using here um

And yeah so I'm using uh 768 Dimension embeddings and then you're going to click on embeddings here and um if I show you this we could add you know coher embeddings AWS Bedrock aour uh Google Palm Gemini hugging face m mistal Open AI for this case just especially just you know in uh spirit of the video I made yesterday on showing you how to connect olama with NN we're going to use an oama embedding because why not right so I'm just

Using the nomic and bed model if you want to know how to set up AMA on your na instance and use it check out my video I did yesterday Link in the description then for the document you're going to just use the um I'll show you what this looks like I just want to make this very simple for you guys the default document loader there other options to get Hub document load loader we're going to use default one right and it's

In Json for a type of data right and then we're going to for text splitter I'll show you there's different options right but we're going to use a token text splitter or token splitter and chunk size I'm doing 512 by and then the overlap is 100 okay um so yeah that's pretty much what this is like the insertion uh of the into the vector store all right so there's really two parts of a rag system This Is Us inserting data into

The vector store you don't know how a rag works I'll quickly go quickly quickly go over it so we have that up we want to be able to do retrieval augmented generation on um we're sending it to our Vector store then the embedding model is embedding it into what dimensions and we're using a token splitter to chunk it into specific sizes so five 112 tokens and the chunk overlap not going to like overe explain everything I just want to quickly explain

This but it's chunk overlap is 100 okay from that point on then we're able to do the actual retrieval part or the um uh query part okay so this is how that this how part one works and like I said keep in mind this is where you can like you could set a node here and like send it to another workflow um like this is where you could add uh use like tools and stuff I'm not you know I'm just saying

This quickly if you don't know what I'm talking about don't worry I'll do more videos showing more in-depth stuff but yeah I just trying to spark some uh of your imagination as to what you can do with this okay so this data could come from anything just that all right so now we're doing the right side which is the C uh querying so I just put a SP a basic chat uh trigger right here that we're going to use to test

It after and then this is our question and answer chain so if you go to Advanced AI options I love n they already have like all the Lang Chang Advanced AI type stuff um you would select question and answer chain right here boom next we're going to use our model which in this specific case I'm just using the open AI chat model gpg 40 you can use anyone really doesn't matter you could use a llama you could use hunging pH you

Could use literally anything you could use anthropic okay now for the retreat doing a vector store okay and what Vector store are we using of course we're using uh zap now oh one other thing I forgot to mention is important in your Zep uh on the left and the right you want to make your own collection name so you're going to name your collection name you can name it whatever you'll see uh once you like if we go here I'm going

To delete this um just because we don't need it right now and as you'll see there's no collections here that's okay because once we upsert it in there it's going to create that collection name with whatever you put here so for us we're just naming it test and you're going to want to put that same name over here so we got test you're going to want to put the same embedding Dimensions that you used and then you could literally just copy

The like you could copy the EM embedding model over here too but um it's or just set it up it's really simple put a Lama same thing no nomat uh nomic uh embedding model we're using right here nomic can bed and that's pretty much it so just to recap one more time just to make sure you got everything set up you got the vector store on the left embedding model default data load over the chunk size to uh CH actually token

Splitter is the chunk size and then you have your collection name here you have your collection name here the embedding model here your uh AI model here Vector store and then um yeah that's pretty much it and yeah you can like expand upon this um make it more advanced of course add it into your you know nend workflows but anyways let's go ahead and just test this real quick going to Boom but now we're actually going to upsert the data in

So set the set the field we just testing it boom boom boom you can see it's embedding right here give it a second okay so it was a success if we go into our Zep right here and we just refresh we'll see that we got the test collection okay so test collection we got five documents five embeddings index Etc embedding with we can see all that good stuff so we're good now let's test the chat so the retrieval side of things

Um what is the name of scene three and five from Act One 3 1 three and four all right and here are the acts right here so scene uh from name of scene three and five from 1 three and four so we have act one three is this room in capet's house and then five is a hall in capulet's house and then Scene Three we got or act three we got Scene frier Lawrence's cell and then five an open gallery boom

And then act five Scene Three is Juliet's chamber and then scene five is Juliet's chamber Juliet on the bed so it it did it correctly and we can even see the logs for the last message we can see Vector store retriever can see all that and then input output we can see the Zep what happened there as well as the open AI right here as you can see it pulled this chunk and got the information and got it in the output

All right guys so hope that helps you hope that gives you some context and some um some good uh you know learning and uh implementation that you can do right now on how to set up a rag using zap you have any questions whatsoever at all regarding any bit of this process first off check the link in the description to go over everything make sure you follow the instructions correctly if at that point you still ever having issues let me know

In the comments down below and I'll do my best to assist you um but anyways guys if you're new to the channel we upload videos every single day on AI automation business growth AI coding um Marketing sales go high level uh all that good stuff so if you're new to the channel make sure to smash that like button smash that subscribe button if you got some value here definitely appreciate it definitely helps out also too guys um if you haven't already

Joined our free community stri community.com our free Facebook group and Discord channel will be linked down below strid community.com other than that guys I will see you in tomorrow's video keep hustling keep grinding and of course guys accelerate your stride take care