FREE: Any Ollama A.I. Model On n8n Self-Hosted!🦙🤖 Llama 3.1 - Docker, Coolify, Portainer🐋 (Make.com)

What's going on everybody Josh pook here and in today's video we're going to be looking at n8n and more importantly how you can connect n8n with local models such as I mean leveraging o llama so you can use llama 3.1 you can use mixol you can use 53 you can use quen 2 you can really use any single model on olama in your n8n workflows when you're building out NN agents whatever the case may be let's dive right into it [Music]

All right guys so I'm going to walk you step by step on how to specifically do this because I've gotten some questions um if you're new to the channel we upload videos on AI automation business growth Etc and maybe if you're not even new and you just subscribed recently you maybe even haven't seen some my other n8n videos so I'll give a brief intro to n8n and I'm not going to cover everything all right there's a few Tools in here that

I kind of assume that you may have already know about or you can check out some of my other videos Linked In the description if you want to find out more about that all right I have full videos showing you how to self-host naden you could do this self-hosted um and it's it's very simple also too guys um so there's a yeah anyways there's a few tools let me just get right into it okay so first off this is basically just

A very simple workflow I just threw together it's really not doing anything as you can see it's um but I just wanted to give kind of the framework for an AI agent now if you're not familiar with AI agents and actually now I remember what I was going to do we're going to go to naden I'm just going to quickly go over this so if you're not familiar with naden secure AI native workflow automation blah blah blah you can see here

I'm not going to read through the whole entire um website but 400 plus Integrations it's essentially if you're familiar with zapier or make.com it's an alternative to that you can go on their Cloud uh platform here there's price whatever I personally don't care for this unless unless maybe you're an Enterprise and you just really don't care and you want to you know you need specific features here really doesn't make sense just self-host it and if you're wondering oh is it how's

It compared to zapier or make I mean this is my opinion some people I know there's a lot of zapier or make.com fan Fanboys or fan people whatever I like those tools too they're great but my opinion n 's way better every like every single metric it's just completely dominates it it's funny cuz like so many people aren't even using it um which is fine but yeah anyways that's that's end then check on my some some of my other videos on

How to self host it also just so you guys know this doesn't really necessarily um 100 I mean actually this could be very easy if you really want to self is super easy I've spec basically said this in my video you can literally just go to coolify you can run one command and then within coolify they already have a pre-made template for nadn so it's really like no excuse okay you just need a VPS yes and yeah I mean you can

Self-host it on your own local desktop but I just would not suggest it um but anyways so now okay so I've done videos on NM before building agents and people ask how do you connect AMA all right it's very easy to connect other models it's really just like you uh enter your credentials but how do you connect AMA so first off um a llama is basically has a Docker image so that's there's I mean there's a few different ways you could

Do it I'm just going to show you the way I did it right and uh we're going to use the docker image just because this way I did it makes at least for my opinion it makes the most sense for me and my specific situation it probably make pretty good sense for a lot of you guys um but anyways so what basically we're going to do is we're going to install a Lama in a Docker image on our server and then

Long story short we're just basically going to swi add the that Docker container to the same network as our n8n instance for like assuming that you're using n8n within Docker as well and if you do do it through coolify and everything you're going to be doing it that way okay and then what we're going to do is we're just going to run the uh we're going to run well first off basically just run the docker Pole right here then we're going

To run this Command right here and here so I'll walk you through it so okay so you're going to want to SSH into to your VPS and then you're going to want to run Docker pole AMA so literally link for all these everything I go over will be in the description down below so you can just copy and paste follow along very easy Docker pull command so you're pulling the image right you can do this right off the get-go Docker pull

The llama and then you're going to run now this this is dependent based on what server or you know computer you're using um for me the VPS I'm using right now it's just what a c we're using CPU only but if if you're using something that has an Nvidia GPU then you just want to follow um I mean there's there's different options right here like AMD AMD GPU um so if you do have that uh if you're if you're have a

Nvidia or AMD you're going to want to follow probably these instructions right here and you just go about running these commands configure um everything this way uh with the Nvidia container toolkit Etc but let's just say for example most most of you or a lot of you probably just say if you want the basics um you're just going to run this CPU only command so you can literally just copy it and paste it in your terminal run it and basically what

That's going to do is pull the container and then run the container all right so now the container should be running on your server right now um not like yeah so okay so the way I did it is I just used painer just because for me I like painer it's like a nice gooey for Docker I'm not going to go over like how to install painer for this um but I did do a video on how to install painer you can

Check it out on my channel if I I may have a link in the description for it if you want to install painer just it makes things a bit easier but I'll also go over how you could do that with um how you could do it without painer it's just you'll just have to run a few more command lines but basically once you have uh yeah so the next step is the painer thing so let me just say um on my

VPS how do I do so all right so this is literally just what I do just chat GPT it or clot it whatever and what you're going to do is you're going to do Docker Network LS this will list your networks okay then you're going to add the container to the desired Network so uh use Docker Network connect command to add your container to the network so you would have to run Docker PS Docker PS is going to list out all

Your Docker containers and I'll show you kind of what that looks like um so to kind of show you it would look something like this so you can see this is a container ID and this is a Docker image um so you could either put your let's just keep it for you could either put the container name or the ID here and um basically and I'll leave these commands in the description too so you can just copy and paste it so

Docker Network connect um and then you need the network name and this network name is what you would get from the network name right here when you list the network name now the network that you want to connect it to of course is your n8n instance so this is assuming that you already have n8n installed on your server right you could this could be through coolify this could be really anything but let's just say it's coolify you're going to um get

That you could if you're in coolify you can see the network that it's account uh listed to you can see that as well in painer um like if I go to painer here you can see that this is my o llama instance right here this is my ol llama container and you can see if you scroll all the way down so you go to containers and then you select your AMA container um if you scroll down you'll see the networks that

It's connected to so as you can see it's connected to my bridge Network which is just my my Basics my server and then it's also connected to this random you know Network right here with random um things that is the n8n instance okay and yeah so what you would do just in painer is you would just click this drop down to select a network and select the network so it's a lot simpler I guess and as from a gooey standpoint and

Then you're just going to join the network there okay so I hope I I hope I somewhat explained it okay for you guys you have any questions let me know in the comments down below but like I said all the uh commands that are necessary for you to do this let's just say you're not using painer let's just say you're doing it straight through the CLI command line all the commands will be down there so just to go over it one

More time you have an installed you have a installed n n 's in its own Docker container and then o llama is in its own Docker container you just spun up and you're going to Docker PS to get the list of all your containers you'll be able to see the ID of your uh of your llama container and then you're going to um get your Docker networks Docker Network LS to see the docker networks and then you're going to um you're

Going to basically put the network name right here and the container ID right here okay whatever those are for your specific case Okay so once you do that your o llama should be connected to um the same network as your uh NAD instance okay from that point on um what I did is I restarted both um both of the containers so you could go into coolify restart the container if you if like I'm not going to I don't want to like

Go through every little nitty-gritty thing like these are basic things if you don't know how to restart a container just literally chat GPT the command um because I don't want to make this video too too long I just want to get get you guys to the um to the point as quick as possible so yeah so now that you have the network the olama connected to the same network as NN you're going to run um basically you're going to run this

Command which is um which will be in the description down below um so Docker X exact and then - it and then this is going to be the ID of the container so that this is going to be the ID of your um of your olama container so you're going to H have the AMA container ID here and this is basically just allowing you to run a man within that specific container and what we're going to run islama uh and then

Really this is where you run whatever model you want to install so for me I'm just doing llama 3.1 so that's AMA run llama 3.1 but of course you can go do a llama and install whatever model you want okay so um let's see so yeah I did and this like I said will be in the description down below if if anything's confusing please just you know look at the description copy the commands you should be good if you have any

Issues let me know and I'll do my best to assist you um okay so now that you ran this you should have olama installed on your um on your AMA instance right or your olama container from that point on we can be in n8n which is right here right here and then we can go to add a node for example let's just say like I mean this is what I did I added an AI agent from here you can connect a

Chat model okay and as you can see I just selected AMA chat and then of course you would go you would go to your um credentials right here and your base URL so it would say Local Host um and then the port of AMA 1143 or 11434 but you're going to want to put the IP address of your Docker container so let me just ask chat chpt how do I get the IP address of a specific Docker I think it actually

Just shows it um container let me see I think Docker [Music] PS okay well let's see so I guess I think okay well this this uh you can list all running containers so you can run Docker PS to obviously get the ID and then you just run Docker inspect and then the container name or ID so you're going to inspect your uh AMA container to get that IP address all right I'll put that in the description too um and then that's

Going to give the IP address of that specific Docker container running on your server and then you're just going to put like HTTP h pgp colon that same IP address and then the AMA Port as you can see here I got olama uh the model llama 3.1 the latest set 8B model um that's installed on my server this is the only model right now this is just a fresh um in install of AMA and as you can see now I can

Use the chat model or theama chat within my n8n workflows all right this obviously unlocks a bunch of stuff you can do if you if you're a fan of open models as well as two guys like when you're doing any in workflows sometimes there's things that you know just aren't necessary to run Claud Sonet 3.5 and spend x amount of dollars or run gpg 40 what uh whatever the case may be right you want to run a local model maybe for

Privacy purpose whatever right so this um hopefully helps you guys I know a lot of people were asking me about local models with an n8n how do you do it because if you're just running it on your desktop and it's just the um it's not like a a live link uh base URL that you can put in so this is how you do it um there's there's other ways too but this is the way I'm showing you this is a pretty

Easy efficient way to do it in my opinion but anyways guys hope you got some value from this video I'm going to be doing more videos on nadn as well as other automation stuff so like I said if you're new here we upload videos every single day on AI automation business growth uh AI coding developing whatever the case may be I'm trying to get you guys the best Cutting Edge information I possibly can so um yeah I've been working on a

Lot of cool stuff got a lot of cool stuff planned but also too if you guys haven't checked out our free community stri community.com link for that will be in the description down below our free Facebook group free Discord Channel going to be put putting a lot lot of cool free resources in there but let me know if there's anything you guys uh are struggling with with this video I'll do my best to help you guys in the comments down below

And let me know if there's anything specific you guys want to see me build out an N an an n8n and let me know if there's anything specific maybe another integration or something that you are having trouble with or a concept you're trying to wrap your head around with with n Ann um I consider myself pretty proficient in like bu and having a lot of experience building out different use cases for clients myself Etc so um anything you guys are struggling

With let me know and I will do my best to make a video on it and help you guys out as best as I can um so yeah other than that guys hope you enjoyed see you in tomorrow's video keepy hling keep grinding and of course guys accelerate your stride take care