FREE: Meta Llama 3.1 70B & 8B w/ Groq on Self hosted n8n using Coolify 🤖🦙 (Make.com Alternative)

What's going on everybody Josh pook here and in today's video we're going to be looking at how you can actually use llama 3.1 meta's new most capable opsource model to date with n8n if you're not familiar with NN it is a secure AI native workflow automation tool it's an alternative to tools like make.com zapier but in my opinion it's a lot more powerful all right I'm going to show you guys can set up an8 and selfhosted all right so you're not

Paying anything except for your server and then you can save money from tools like make.com zapier and you're going to be able to leverage these powerful AI automations let's dive right into [Music] it all right guys so if you're not familiar with llama 3.1 just a quick recap and I've done about two two or three videos in the last couple days about meta's new tool that Mark Zuckerberg released and kind of took the industry by storm well to summarize llama is

Quote unquote an open source model some people say that you know that's debatable whatnot um but anyways I guess because I guess the the weights and whatnot and and this and that some people say but anyways um up until now open sources are always been kind of trailing behind uh closed source SCE and what we are seeing now is that this is kind of that point inflection point where open source is uh catching up and now we have meta AI which

Is the first Frontier opensource Model and I'm just going to pull up a graph right here all right so if you take a look at this graph we have closed Source versus open source uh weight models and you can see open source right here or closed Source right here now we have Claud Sonet 3.5 GPT o Gemini uh Pro 1.5 Pro Gemini Ultra and now we have uh open source and we can see it was trailing behind and now we're at

Llama 3.1 405b 70b 8B and it's closely catching up right so we're going to see um you know we did do some tests with llama 3.1 is it the best model um that I found no for like certain things like for coding um there's definitely better models and for a lot of things to be quite honest but it is an open source model and it is definitely um high quality for you know its category and it definitely does um you know

For cost and everything it's definitely very powerful and I am going to be using it more and more but and as well testing it with C with different scenarios to find its strengths to see where it actually sh shines you can see here's some benchmarks all right um just some basic ones I'm not going to go through it all if you want to check some of those check out my other videos I just did but we can see here llama 3.1

405 comparative to models like gbt 40 Claude 3.5 Sonet I still say Claude 3.5 Sonet is the best for coding what I found I usually get the right uh responses with that um uh and GPT 40 is pretty good and uh yeah and there's some other good open source models that I find are even better than llama um such as deep seek quen uh to and anyways so llama 3.1 benchmarks for the 8B model so this is a model this model

Right here the 405 you're not going to be able to run locally unless you have a a beast of a computer um but the 8B model is something you can run with AMA I show you guys how to do that in my video yesterday so if you want to see that check that out but here is the open source model comparative to other ones as well as gbt 3.5 turbo okay so that's uh llama 3.1 you can also test it out

On meta's website I'll leave links to this the article and all that stuff in the description down below now what we're going to be doing in this video is self-hosting n8n I mean you could also use their self-hosted version but I recommend self-hosting you can get a VPS to self-host it um I I'm using a contabo VPS so I find them pretty cost effective but you can use anyone um you can use header you can use whatever wherever you get a

VPS technically you could host it on your um your own local system but in terms of when you're actually using web Hooks and and uh deploying it on the web you're going to want to use um not your local system all right so anyways what we're going to do just to make things simple is you can do this any way you could follow NAD 's docs which I'll leave a link to naden and I've covered many naden videos but uh you

Could just selfhosted directly on your server we're going to use coolify though so link for that will be in the description I have many videos on coolify you just simp simply run this Command right here it's one line of code curl and then the the install bash um link right here it's going to install coolify once you do that you're going to have coolify like this you can uh you can set your coolify up and like connect your domain I'm not

Going to cover that I've covered that in many videos um check out my coolify videos if you want more info on that but you're going to go to projects you're going to add a new project you can call it whatever you want all right I'll just do this n8n one then we're going to go to our produ environment we're going to create on a new uh resource and then you're going to scroll down and you're going to see that there's already

Preloaded um n8n images right here so you could either do NM with postgress SQL or just normal n8m all right I already have an NA environment deployed so I'm not going to deploy this one I'm actually just going to delete it but all you would do is you literally just go into the resource like this and you would just click on deploy all right it deploys out of box um some coolify apps you need to do certain configurations and whatnot for

Nadn it's very simple you literally just deploy it you can change it to a subdomain pointing to your IP of your server and then you can self-host it on a subdomain of whatever domain you want or a main domain what whatever you really want right so I'm just going to go ahead delete this and we're going to go into n8n okay so before we get into n8n you're going to need to basically if you're not familiar with n it uses something

Called nodes and it has native AI agent nodes all right I've done many videos on n8n showing you how to do advanced stuff like uh or I guess somewhat Advanced not too advanced still pretty simple but like stuff like you leveraging AI connecting it to air table you know I even showed uh videos on how to query um create a uh a rag system using tools like uh Zap for memory or vector stores as well as tools like super base and

I'm going to do more of those as well and I'll probably even maybe do some with llama and show you how to do that but in this video I just want to show you how you can actually use the Llama model because if you go to like if we just click on model here you'll see that there's anthropic there's a azour there's uh mistol there's open AI there's Gro so if you're not familiar with Gro that's what we're going to be

Using and you're going to have to create a Gro account all right so you can go to gro.com I'll leave a link below once you create an account and you can test out the models here too chat with them Etc but uh you're going to click on grock cloud once you do that you're going to go to API keys and you're going to create a new API key okay once you do that you're going to go back to n8n and then

When you select your grock uh your model which is grock you're going to create a new credential and you're going to paste in that API key once you paste that in it will allow you to save it and then you'll have your Gro account here from that point on you can go ahead and select this drop down and you'll have access to a bunch of GRS open source models that like Gemma uh llama 3.1 uh llama 3 point so we have

The 70b versatile model and the 8B instant model and we then we have some of the older models right so in this case we would use um we could use I mean we really could use this this one or this one now it currently doesn't have the 405b model at least um I guess not on my account I don't know if it's maybe in Alpha for maybe some users or whatever the case may be but anyways this is the 70 model

Is okay for for you know what we're using it for right now also too guys if you um if you didn't check out my video yesterday I got I showed you how you could actually use llama uh 3.1 within VSS code using a tool called um continue dodev so you can literally set it up in your code base and start asking it questions and you could filter it to uh llama 3.1 40b so you could actually chat with the 405b model

In your own code base so if you haven't seen that video check that out but once we have our grock chat model set up here what all I did is I clicked here and I clicked on Advanced API modules uh or nodes and then I just clicked on AI agent all right once you click on AI agent you'll see here there's three options so there's the model which we just added for grock there's memory and tools so I'm not going to

Do memory or Tools in this video but just like if you want to do that I've done videos in the P like I've done videos naden uh if you search on my YouTube probably a couple weeks ago naden rag system using supera base using zap I show you how you can actually connect memory all right so you could use Zap for memory for example or you could go to tools you could go Vector store you could um you know put some

Info here you could go Vector store you could select super base or zap there's other options too like pine cone cant um but yeah I like super base and zap and then you can select like a model for embedding and and do all that that's for what's called creating a rag system a rag system is retrieval uh augmented generation so basically it's uh essentially a system where you can put your info you can preload like a n like your knowledge into

Like a database like superbase and then it will uh you can use an embedding model to embed that data so it will essentially take that D data and put it into um like dimensional uh dimensional numbers and whatnot and then um when you query the database it will turn that query into an em embedding and then it and then um it will use mathematical calculations to determine the similarity or the cosine or whatever between um that embedding code Dimensions as well

As the ones that are in the database and then it will be able to use that to give you the higher a higher quality output if you want me to do video a video on that uh showing you guys how to do that with llama 3.1 within N8 let me know in the comments down below but um essentially you could just use Gro right here and then watch my other video on that and just uh change the the model that I

Show I believe I used open AI uh embedding so you just use llama 3.1 with that to do so um anyways now that you have this you got the very Bare Bones of what an AI agent now right now you could customize this so you could add other nodes like you could add air table I always like using air table and I usually would add air table at the start like what I would personally do is I would do like uh

I would go add node and I would do a web hook um so add another trigger I would do a web hook um and then yeah so I would do a web hook and then I would go a new air table and then I would get a record so I usually like doing a a air table a web hook and then I send the record ID from Air table and then I usually just get the record ID um with this get

Node and then I just uh put that to my air table AI or to my agent within NAD and um this is good like if you just want to pass data from like a like a a no code database like air table all right so it's a that's just a simple method I don't I don't want to like try to if you don't if you feel like I'm kind of going over the all over the place I'm just kind of telling

You guys different things I'm just trying to give you guys different ideas and options for what you can do with this but just the bare bones basic right here we have a chat so I just use a chat trigger and if I go here and I want to test the chat you can see here let's just say hi all right hi and you can see if it works we're chatting with grock's llama model it's going to say hey how can I

Assist you let's say hey is this llama 3.1 uh I am an AI model I'm not specifically I'm more General language I for I've noticed that la like this llama just doesn't sometimes it doesn't say it's like llama 3.1 um is this made by meta sometimes it does sometimes it doesn't like I've asked this question in my other videos you're curious about the technology behind our conversation yes I'm using so yeah it's it kind of like hides it for some reason

Sometimes but anyways you can you can definitely see it's meta's new model and we're using Gro if you're not familiar with Gro they use what's called um lpus instead of gpus and uh it's it's very a lot faster chips like you can get very quick response times low latency so I really do like Gro and I believe they give you like some free credits and uh or I think there is like it's pretty much free to a certain degree um there's

You go to their settings here billing yeah so there's the free plan the paper token is coming soon so you can do this essentially for free at the moment but there is uh low rate limits so if you're going to be using this for like a big NN buildout or whatever the case may be um you're going to be rate limited so I'm still waiting for this paper token plan to come out and then there's the pricing for all that stuff

But Rock's really good nonetheless you can do this for essentially free if you're using it um if you're using this on on a small scale for certain things then um yeah you can get fast inference time and free as well as uh free self-hosted n to pretty much build anything all right n is such a powerful tool I've done many videos on it I'm going to do many more I know a lot of you have been asking for it but um

Yeah at this point you should be able to have your own n instance uh hosted all right using coolify as well as connecting uh llam llama 3.1 meta's new model into naden and um and then you can build your custom workflows from that point on you can really do anything naden has many different nodes you can connect to and um really get creative with it and uh leverage it right so anyways guys hope you got some value from this video if

You're new here and you got some value make sure to smash that like button and subscribe to the channel we upload videos every single day on AI automation uh business growth Marketing sales Etc so uh stay tuned for that also too guys if you're new here and you have haven't already joined the stride Community what are you doing it's free we have about 750 to 800 members on our free Facebook group and free Discord channel so link for that will be

In the description if you want to network with like-minded AI uh enthusiasts entrepreneurs developers coders marketers business owners Etc myself all right if you want to bounce back ideas back and forth or you have some ideas recommended for a YouTube video join the link down below stri community.com and also too guys if you haven't already checked out my book client acquisition code if you run a business or if you're someone that's building stuff with AI you want to be proficient at

AI prompting Marketing sales whatever check out my book it took me a long time to write this all right uh it's 500 600 Pages um and yeah so it's client acquisition code link for that will be in the description Down Below guys other than that I will see you in tomorrow's video guys keep hustling keep grinding and of course guys accelerate your stride take care