FREE: Open Source AI LLM API Cloud-Hosted 🌨🤖 Groq AI API - Mistral & LLaMa AI LLM

What's going on guys in today's video I'm going to show you how you can use open-source AI models like llama and mistal without hosting them on your own PC or system how you can have them hosted for you in the cloud and use their API in your applications in your software in your workflows automations whatever the case may be absolutely for free this is going to save you a bunch of money let's dive right into it [Music] all right guys so

Why did I actually come about looking for this solution or really just doing this um well first off I use open ai's API I use perplexity API I use you know claud's API I use all these close Source models API and they're great I love them they're very useful ET but um it's definitely good sometimes to Leverage The Open Source models too and for specific tasks you may not need the full power of a GPT 40 or a claw AI now

Don't get me wrong at the moment those models still are top tier above the open source models but the open source models are really good and for certain tasks for example in my specific case maybe if I had a bunch of text that was just really just eating up my tokens with other API models and you know I'm ending up spending a couple hundred bucks a day just on this one little task per se when maybe the task is you know

Just summarizing a block of text or really just you know categorizing something right that is just you know a bunch of text a bunch of tokens for really no reason it's just using some basic uh you know you know intelligence right so for that specific case you know something open source or free or low cost would be useful right and I want to say this isn't a perfect model as of now right there are some rate limits that go about um

Using this model and it's not something that's necessarily new it's a newer it's somewhat new but I mean people have heard of it I've done videos in the past but that I haven't talked about using the actual API and it is through Gro so grock with a cute not grock with a K not Elon musk's grock but Gro with a Q and I've done an in-depth 45 minute video going over like their new Chips when they first came out and how

Fast they are and these are basically the fastest chips um pretty much in the world that we know that are open to the public and anything like that for you know these AI models and it's super fast like you can go to gro.com and it's really nice because yes you can use their uh models like llama 3 8B Gemma 7B uh all these models right here Mistral for free and you can chat with them and um you know you can see

Here I was asking at about the API here about some of the rate limits and stuff like that so like I said there are some limitations here there is a paid version coming um which will be good for those who want higher limits but um yeah I'll show you how you can leverage their free version and um you know uh it is definitely useful for specific use cases but you can see here you can test it out like hey um hey

Uh [Music] when get or write me a story on a bear that loves honey make it funny right and I'm just typing something here just to show you guys how quick these this AI is or these chips on really this is obviously boom so you can see the speed here speed insights input tokens output tokens tokens inference time um you know Point uh input seconds z uh 58 output time 044 so in total 1.0 uh 02 seconds tokens per second input

Output Etc right so you can see that those that uh those details there you can you know change it to bullets expand sentence Prof so this is cool you can actually tell it to improve upon like a Chad and you know make it more neutral tone friendly professional copy it uh scroll to top Etc there's like um Discord Channel and all that good stuff but um basically what we're going to be using is over here you actually have to go to

Wow. grock ww.go with aq.com and that will show you kind of like you know they have Enterprise access here you could apply for you don't you're not going to really be doing that unless you know you're big Enterprise um price right here so Gro guarantees to be any published price per million tokens by published providers of the equivalent listed model so that's really good to know um and other models such as mistl and code llama are available for specific customer requests

And your inquiries here and here are the models llamas 370b 8K context length um so yeah also limit I mean these are open source models so they're not the highest contact length in terms of like what we see with stuff like Claude or uh GPT 40 but you know you know here's definitely it's definitely cheaper so yeah um mistl uh 8times 7B Smo 32k contact length so a little bit higher there um if you need a higher contact length and then

Llama 78b AK contact length Gemma 7B AK contact length and you can see the pricing here and they also have whisper large uh version three where you can see all the costs associated there and you may be saying Josh you told me that it was free or there's no cost well currently if I go to you're going to click on grock cloud all right here you can see like their playground where you can access the models here change temperature max tokens

There's two modes you can enable such as stream the returns the completion in a stream rather than waiting for the completion to finish and returning it all at once there is no support for Json mode when toggling stream Str when streaming is toggled and then Json mode so this basically uh guarantees that the output is a Json object um top P speed and then stop sequence if you're using that but um here are the documentation we'll get into that in a

Second API keys so this is where you would create your own API key right so API Keys is very simple it's obviously just where you go create your API key and that's where you will be using that API key so we'll be using that in just a second here but if I go to settings and building you'll see that as of now there's only free plan um the pay on the pay per token on demand plan is coming soon and you

Know this includes higher rate limits priority support to be honest even though like a free option is great once this comes out if I'm you know really using Gro I'll definitely use the pay per token plan just because I don't mind spending money and I'd rather just have higher output higher rate limits but you know in terms of like small use cases on applications like you can't build a whole software with lower rate limits Etc depending all depends on your rate

Limits and all this or how much rate limits you would need but the free plan's really good um especially just for using it for specific use cases Etc so here you can see that the on demand pricing that we um were looking at on the other page here so for the Llama it's uh 59 cents per million tokens on the input and then the output is 79 uh mistol is 24 on the input and 24 on the output llama is second

Llama is 5 cents on the uh input and uh 8 cents on the output 7 cents here for Gemma 7 cents on the output as well and then 3 cents per hour transcribed for whisper okay so that's the pricing once the on demand does you know have that feature come or that plan come out as of now you got to stay on the free plan if we go to the documentation here you can see like if you're using it within your

App or whether you know you want to use uh J on curl JavaScript python Etc it has documentation for all that mainly I'm going to just be showing you how to authen authenticate it in something like well really anything but I'm going to be showing you specifically an n8n same concept applies to make.com or really anywhere but you know this is just connecting to the API doing a completion Etc so you can read up here I'll leave the docs in the

Description down below you know different information on errors change logs examp features so chat completions open AI capabilities speech to text tools integration so grock libraries um Lang Chang llama index so pretty cool stuff there and then rate limits um are shown here rate limits so yeah the rate limits and then um if we go to API reference this is where we want to go and here we can see our different options our different endpoints so really the main one that

We'll be using is I'll show you well chat completions create a completion there's also ones such as list models which I mean we don't I don't need to you just use that maybe you do for some reason and retrieve model which we don't need to do we're just going to be creating a chat completion and it's very simple if you've you know connected to apis before done API requests very very simple um we're doing a post request to this URL right

Here um we can see here the different options we have have which is frequency penalty function call uh this this is uh um no like you don't want to use this anymore appreciated and then um let's see Max tokens um so that these are the ones that are required obviously message and model and then you know you have all these other options you can look at them and um it shows you the sample um request here as well as the sample

Response and you can change that from python to JavaScript just to make things really simple like I especially when you have like these AI tools at hand such as especially now Claude Sonic if you haven't seen Claude Sonic's coding capabilities and all the crazy cool stuff 3.5 of course check out some of my other videos I did some cool videos on that going over it but just to save me time like I I know how to connect to API I I

I could easily just read this and figure it out I mean it's very simple it's just right here but just to C like even though like it's the exact details are right here I always just like to go which I did to um I like to go to like a AI model so I did this with um quad just because it's the best with uh coding and I just said read the AI API docs on chat completions I want to make

An API request to gu rock using Json explain to me everything uh what I need to do and the Json format so I just wanted it to give me the best Json format um just so I don't have to you know look and see what I want to add even though it's very very simple um and then it just said and then I said wait here's the actual the real the reference page so it gives me the uh the endpoint which

We already knew but here it is authentification so very simple you're just going to use Au authorization and then Bearer your API key this is if you're passing it through the header or whatnot um for n they actually have a kind of native uh integration so you just put your API key I'll show you that in a second then we're doing a post method um for the request and then this is like sample Json and it just basically told say shows

The key parameters here that we could change such as model message temperature max tokens top P stream um and then yeah that's pretty much it it gives some additional options Etc but we are going to let's see so let me just show you how this looks like very very simple you can this applies to whatever platform you're using whether it's NN whether you're using the command PR uh command line or you know you're using actually building out application it's very similar

Uh just to connect to the API of course so here I'm just adding an HTTP request and we're doing a post method and we're putting our URL grock for/ openai track completions now for uh for for naden if you're using NAD specifically you can go predefine credential types and then you can just select Gro or you can type in Gro they have it here and then you want to go here actually no so you would click on this drop down and

You would create new credential then you would put in your API key API key like I showed you is available right here you just go here create a new API key submit and then it will give you that API key you must copy it before you exit it or else you won't be able to see it again and you paste it over into naden or whatever you're using um for like I said for something where you doesn't have maybe a native

One like this or where it doesn't already have predefined credential type you're just going to put authorization like say if I was sending a header here I could just do that and put authorization I'm not going to yeah authorization I spelled that wrong and then put Bearer and then my token key would just go here and that's you I could do that as well but since uh Na and has it uh predefined already I don't need to do that so I'm

Just going to delete this and we're not going to send any headers um we're just going to send a body and the body actually I only need I'm just going to send this so let's just see how this works um this says just say text test so here we we go we sent it and we can see the output so we get the chat ID object created model choices so message assistant test is there anything specific you would like me to

Test or elaborate on I'm here to help with questions you may have um just say test see test oops say oh gosh just say test say this word don't sit say anything else see if we can actually get it to just say test okay perfect I feel like uh close Source model even if I just said just say test it would probably say let's see so go here let's go just say test it's probably yeah just going to say test so

Obviously we're using open source models they're not as good you kind of have to very be be very specific with your prompts that's one thing I'll say when you're using these open source models you're not have to talk to it more of like it's hey it's a it's a toddler it's a little kid here it's you know you're not talking a Mr Big gp40 or the Claude Sonet 3.5 that can completely understand everything and obviously they're not perfect either sometimes you

Think oh this you're dumb how do you not know this but you know these are like you're basically talking to Little Child AI Bots or models I mean yes they're still very good for open source and open source is making a lot of progress and you know I'm excited to see where this goes in a year from now but just keep that in mind still they're not as perfect if you're used to using gpg 40 or son it even those Z's

Aren't perfect either but so with your prompts be very specific anyways you can change the model here um if you want to CH different models and then there you have it that is pretty much it guys I wanted to just share with you how you can actually use grock in your builds and you know I'm also going to be looking at different Alternatives as well on how you can leverage open source models or models that you know for specific tasks you

May not need the full Firepower of these Mo these like gp40 or son it and it's just really eating up a huge token cost all right cuz yes it can stack up if you're doing thousands and thousands of things every single day and you're dealing with large um text blocks and still this isn't a fullprof strategy just because we're limited with rate limits at the moment um so if you're doing like huge amounts you can't it's not scalable but um it

Is really good you know you don't have to host open source models on your own systems host it in the cloud using grock's fastest uh AI chips so this is something I feel like definitely could be of value to you in your builds and if you got some value from this uh let me know in the comments down below smash that subscribe button we upload videos every single day on AI tutorials automations tutorials marketing tutorials you know running paid ads outbound

Prospecting how to leverage Marketing in your business whatever the case may be so if you're a business owner AI Enthusiast entrepreneur whatever smash that subscribe button to stay tuned on The Daily uploads also too guys if you haven't already joined our free community strid community.com we have about 750 800 members in our free Facebook group free Discord Channel check that out Link in the description also too guys if you're new here and you haven't already checked out my book client acquisition

Code um it's super Super Value packed about leveraging Ai and all these stuff in your business I have a full chapter just on AI prompting and I go in-depth with mega meta prompts all these different things like copy and past word for word prompting strategies Frameworks Etc as well as like marketing uh Facebook ads running Facebook ads how to not get banned running Facebook ads how to set everything up how to do cool email campaigns scale with multiple accounts and thousands

Of emails a day um a lot of stuff I put so much value into that book so go check that out if you're into it and then other than that guys I will see you in tomorrow oh also too guys if you do want help growing your business book a call at executives.com for / apply you can check out some of the other videos on this YouTube channel we help our clients grow and scale their businesses in many different ways if

You if you have leads coming in and you have someone that's booking them autom uh like manually we help some of our clients set an AI appointment C are in their business that will book them directly into your calendar so you can literally just sit back potentially fire that person or get them a different task so you can save thousands of dollars a month we have a bunch of other stuff we do for our clients helping them with systems automations Etc

But if that's some sounds like something of interest to you that there could potentially be a fit then book a call down below to speak to myself and we can see if it there's uh you know potential partnership there executives.com apply other than that guys I will see you in tomorrow's video keep hustling keep grinding and of course guys accelerate your stride take care