NEW: 🦙 Meta Llama 3.1 405B Continue.Dev + Ollama + TogetherAI - Github Copilot FREE Open Source 🤖

What's going on everybody in today's video we're going to be looking at llama 3.1 and how you can actually use it for your coding projects how you can get it set up as a co-pilot either hosted locally or using it in the cloud and this is something that's going to be able to replace GitHub co-pilot completely free so you don't have to worry about paying for GitHub co-pilot let's dive right into it [Music] all right guys so if you haven't seen

My recent videos or just any AI videos in the last little bit about llama 3.1 we'll give you a quick rundown but I did a video um kind of going over this blog post here going over kind of some of their tests that you can see here as well as I did a video yesterday going over um some basic testing to see how the uh model performs just to summarize here uh llama 3 3.1 uh there's three different models there's the

8B model the 70b model and the 45b model with 405 billion parameters all right and you can see the um comparisons right here and this is the first opsource Frontier Model that is on par with Clos Source models like GPT 40 Omni as well as clae 3.5 Sonet so this is a new turning for the a AI industry when we're starting to see open source catching up to closed source and things are going to get interesting here but anyways you can

See the comparison here with the different tests and the evals as well as the 8B model comparative to a lot of the open source models here and I'll leave a link for this in the description Down Below guys now get Hub co-pilot if you're not familiar with it it can do your co- uh can be your co-pilot to a lot of uh coding tasks and you can use an vs code and and whatever editor you're using all right it is about

20 15 20 bucks a month or 10 to 40 bucks a month depending on what you're using and um yeah it's it's honestly not the the greatest it's all it's okay but it's not the greatest so we're going to be looking at how you can actually use continue dodev to do this for us for free and I've done a video in the past about continue. DAV I showed you how to set it up use it with uh another open source model

I showed you a video on on how to use it with quen 2 but in today we're going to be looking at using it with wama 3.1 the newest most uh Advanced open- Source model so you can use this right away in your projects a few different ways you can go about this but you're not famili with continue. deev uh link for it will be in the description down below you can check out their website they have a Discord as well

And you can um yeah you can either download it with vs code or with jet brains all right and there's a GitHub um it's open source so really really cool tool simply to download it all you'll do is you'll just go over for example I'm using VSS code and you would go to your extensions you would just search continue and it will load right here you can see here it is the leading opsource AI code assistant and it is it is

Really good for open source where it's at and uh yeah it's it's it's definitely very useful I've been using it for the last um couple weeks and it's it's it's pretty awesome so definitely uh yeah so download this tool really good uh have it as a extension once you download it as your extension you're going to see it appear on this side I like to drag it over here to the right so you can see it's right here and uh we'll

Get back to it in just a second okay so now that we have continued Dev there's a few different ways we're going to use llama all right there's a couple ways you can use it like I mentioned so one is O llama uh hosting it locally if you're not familiar with o llama simply just go to ama.com link for it will be in the description down below you download olama whether you're on Windows whatever the case may be you download the

Tool and then to install uh the model you just run the command so if you go to models here you'll see there's a bunch of different models there's quenu there's um Gemma mistol mistol just came out with another new uh model so we'll look at that in other videos Deep seat coder all the all the different models ama's really great but of course in this video we're going to focus on llama 3.1 and unless you have some massive machine that you

Probably spend at least like 20K 15 20K on um for your computer you're not going to be able to run 405b probably won't even um a lot of people won't be able to run the 70b model so where going to use the 8B model all right so I'm going to show you how to use the 8B model I'm also going to show you how you can use these other bigger models in just a second but um I'm showing you the AMA

Method first um or just going over it real quick so once you select 8B and you download olama then you're just going to copy this Command right here AMA run llama 3.1 and that's how you're going to install it you're going to run that in your uh command line so if I go here to my command line you can see here I actually already ran it once I ran it then you can see you can actually talk to it within the

Command line so I just said Hey it said how's your day going so far I said what llm is this um it told me about that it's using birt and it's a variation and and whatnot hugging face transform it went into in- depth about the actual um exact uh what type of model is but then I just said hey is this llama 3.1 said yes I'm indeed llama 3.1 uh made by meta and then you can continue talking to it you

Could do forward SL question mark help to see different prompts and whatnot but that's one way you can talk to it also too guys I did mention this in other videos but if you do want to continue talking to it you want a method to talk to um use the llm within your command prompt whether you're in vs code or in your normal command prompt Powershell you can check out shell GPT I've covered it in other videos I'm not going to

Me dive into it in this video but I'll leave the GitHub repo down below as well if you have any questions about that just let me know if you're running into any issues but I did cover that in my other video I did on continue Dev so you can also check that out I show you got to exactly install it initialize it Etc with your API key all right so now that you have the AMA situation the way there's a couple

Ways that you can actually use the bigger models within um continue Dev and I'm going to show you how to set up the L oama as well as these bigger models in just a second once you have it but in this example we're going to use together. a you could also use Gro and there's also other you go to um if you go to their continue extension right here and you click on plus you'll see there's a bunch of different options

So you could really use any one of these like there's I mean there's obviously not open a but like to get some of these other ones also have access to uh llama 3.1 you if you don't know you can just check you can go to their website and see which ones actually have it but you can see here we got different options um so we do have obviously options for open AI anthropic Azure uh mistol AMA which you can use um

The local one uh coherent uh Gro together AI which we're going to use then obviously these other ones Google Gemini Etc right so what we're going to do together AI gives I believe $5 worth of free API tokens so you can just go ahead link will be in the description you can sign up for together AI for free then you just go to your um profile API section you copy that API key um you can also check out their dogs and

All the other stuff with together AI but once you have that you're going to go back to your uh VSS code and the way you actually initialize what model you're going to use is you press on the plus and then for us we will select together AI from that point on you're going to put in your together AI API key once you do that I already put mine in once you do that you're going to select the specific model and for

Us we're going to select llama 3 chat all right now you may be wondering oh this isn't 3.1 don't worry I'm going to show you what you're actually going to do so once you select llama 3 chat there's really two ways that you can use continue Dev one is the chat method so you can get it to chat with either asking questions over here or you can even reference your code in vs code and you can upload certain sections certain I'll

Show you that in just a second but different ways you can talk to it sort of like how you would talk to GPT chat GPT but you can also set set up a model for autocomplete so typically the way I like to do it is for uh the chat method I like to use a higher quality model and then for the autocomplete you can use a lower quality model just because you don't always need it per se like you're not using

It as much as the chat um to generate a big amount of content autocompletes usually just to kind of help you autocomplete things right so we can use the 8B model the locally hosted model for autocomplete I like to use the open source uh models for autocomplete and then we can use a bigger model for the chat all right so in the other video I show you had to do this with uh GPT 4 mini which is also a really good

Um uh model for doing this because it's very low cost and it's also very good all right so once you actually click on the Llama 3 chat you're going to click on the Cog right here to see your actual settings within continue dodev all right so you'll see here I have a couple models this is the model from last time uh which is gbg4 all mini we're going to ignore that you're going to you'll probably just have one here or maybe

Two but um you're going to yeah you'll have uh the Llama 3 model which you just selected and you're going to change it to say this and this will be linked in the description down below meta- Lama for SL meta- Lama 3.1 uh- 405b Das instruct dturbo all right so we're using the 405b model and you can see the provider is together and then you can see the context length and then your UPI key would be here I just remove mine

And I remove mine here as well and then yeah you'll just see this is auto test custom command here you can ignore that the next is the Once you would do that though you would just save it so contrl s and then you would have the chat I'll show you the chat in just a second but for the auto complete once you have llama o llama installed and llama 3.1 installed you can just put for the model here llama 3.1 and

Then change the title here for that you can save it and then you'll be good to go for autocomplete as well so I'm going to show you the chat in just a second um I'm just going to put my API keys back all right so let's go ahead and see if this is llama and I'm just chatting with it clever question I am deed a type of language model but I'm not llama the animal I'm an AI designed to understand the

Blah blah blah blah um is this is this llama by meta you really getting into detail now yes I am the large language model by meta etc etc so to confirm my M andd llama large language model application Etc so you can see here this is the chat function within continue Dev you can go ahead and start coding and you'll see the auto completion you set it up correctly now you can see it's a little bit I guess slow right so

If you you want it a bit faster you can also go ahead down select here click on Gro if you're not familiar with Gro they use the uh lpus and um set of gpus and um yeah there this is extremely fast they have the new llama 3.1 model you just basically go create a grock account put in your API key here and then you would just select the Llama uh model for Gro you can just check their docks and I believe

It's like llama 3.1 Gro something like this would just put that in there into the configure file uh that we were just in the settings and then you would be sa save that and you'd be good to go all right guys so this is a really good setup and a really good alternative to GitHub co-pilot you can start leveraging these One open source models you can also leverage uh the other Clos Source models too if you want such as uh open

AI GPT 40 or GPT 40 mini and anthropics uh Claud Sonet 3.5 but um yeah with llama 3.1 and open source On The Rise it's definitely very um you can definitely start leveraging it in a smart way these open- Source models that are um either very low cost if you're using it through a cloud provider or uh basically free if you're hosting it on your own computer with the smaller models all right guys so if you haven't already checked out some

Of my other videos about llama and you need to get caught up to date about that check out some of the ones I did yesterday and the day before since uh Mark Zuckerberg released it and other than that guys let me know what your thoughts are about llama 3.1 if you've been playing around with it as well as too guys if you're new to this channel um we upload videos every day on AI automation coding uh business growth Marketing sales Etc

So if you're an entrepreneur Tech person developer uh AI Enthusiast whatever the case may be and you got some value here smash that like button and subscribe to stay up to date with the daily content I'm really going to be uh increasing the quality and you know trying to give you guys the best value possible been working on some really cool things here with automation Etc so and I'm going to show you guys more applicable stuff that you can do with

Llama 3.1 in the next few weeks so stay tuned for that also too guys if you're new here and you haven't already joined the free stride Community check that out link will be in the description Down Below guys you got our free Facebook group free Discord Channel we have about 750 to 800 members currently so you can go network with myself as well as like-minded entrepreneurs Tech developers uh coders whatever the case may be all in the group below marketers salespeople

Etc and then also too guys if you run a business and need to grow your business with AI I have a full book on it it's called client acquisition code link for that will be in the description down below I even have a full chapter on ai ai prompting how to get the best prompts like go into detail it's a full chapter and the book is about 500 to 600 Pages depending on the hard cover or soft cover and whatnot ebook

Etc so link for that will be in the description Down Below guys other than that I will see you in tomorrow's video guys keep hustling keep grinding and of course guys accelerate your stride take care