NEW: Llama 3.2 Open Source Multimodal🤖🦙 1B, 3B, 11B, & 90 Vision LLM & Orion Glasses Mark Zuckerberg

What's going on guys Josh pook here and ever since Zuck started doing Jiu-Jitsu and hanging out with Dana White he has been on an absolute Rule and today marks the day that meta drops llama 3.2 their first ever open-source multimodal model today we're going to go over the different models talk about their capabilities and see how they actually compare to some of the frontier models let's dive right into it [Music] all right guys so here we are on the meta llama

Website and I'll leave links to all the websites you talk about in the description down below so you can check these out more in depth but here we have introducing llama 3.2 the open source AI model you can find tune distill and deploy anywhere is now available in more versions choose from 1B 3B 11b or 90b or continue building with llama 3.1 all right guys so just to sum up there are four new models two of them are lightweight and can

Be run on the edge or you know basically with in your device such as computers whatever whatever type of device you have and then the other two are the multimodo models which you're going to need to run uh in the cloud okay and I'll show you how you can do both of those or I'll talk about how you can do both of those and then and then in some future videos we will be testing out these models so make sure to

Stay tuned for that all right so here you can see on device like we mentioned this is basically like on the edge where you can use it in phones computers whatever the case may be so for devs building phone apps or whatever this is pretty cool um and we're going to Contin continuously see this trend of where uh these models are going to be able to be run on your own devices you know you're going to probably in a year or

Two from now we'll have these open source models that are super super powerful more powerful than CLA 3.5 that are basically run on your MacBook right so pretty cool to see and I definitely think you should not underestimate open source if you are because we're seeing the progression so rapidly and uh yeah meta is doing a crazy job with uh a lot of this stuff recently so then multimodal so you can use the 11b or 90b models for image use cases

Such as transforming an existing image into something new or getting information from an image of your surroundings so both of these types of models open up a realm of possibilities especially with them being I guess open source I mean some people say that's a debatable where it's actually like fully Source but uh in the sense of what people are talking about I guess you know compared to something like open AI at least these are you know more open source I know

Some people say that it's not really but whatever for example like with multimodo now image capabilities I mean that's very important for many different things you could build and then like I said on device is very important if you're building like apps or just anything on that you want your users to be able to use on the device and then llama has been pushing a lot this whole llama stack um which is seamlessly build a gentic applications for from a comprehensive

Tool chain they essentially want you to build like your whole entire stack with their llama tool stack and uh we'll get into that in just a second a little bit more but it's interesting because they're kind of like we see they're kind of I guess like late to the game of AI comparative to like something like open AI so they're coming with the approach to just light the ground on fire and basically open source everything to you know comp compete with

Their competition right so it seems to be doing pretty good for them right um and then the Llama stack so a stream developer experience build faster deploy anywhere so you can see different stack devs llama stack API they got a new CLI uh the Llama stack distribution um partner API providers models so you can read more about this for Developers for partners and Distributors standard API um they got a SDK all that good stuff and then here's some model evaluations so

We evaluate performance on over 150 Benchmark data sets the span a WI range of languages for vision llms we evaluated performance on benchmarks for image understanding and visual reasoning in addition we performed extensive human reasoning that compare Lama 3.2 with competing models in a wide range of scenarios so you can see 3.2 1B 3B versus Gemma 22b it and then 5 3.5 mini it we can see the scores here for different categories so I'm not going to go through them all

But you can see there you know comparative to the models in this category they're doing pretty good right if you want to check this out more in depth like I said links going to be down below and of course in terms of popularity just with like downloads and stuff llama is leading with the open source AI game they're obviously the biggest company in the space really doing something like this on a large scale so uh yeah it's pretty exciting to see

What they're up to lately uh in terms of Open Source now they do have a more in-depth blog right here which is a 15-minute read so obviously I'm not going to go through all this but I would definitely suggest checking it out for more in-depth um review on these models so it's also pretty cool because llama 3.1 1B and 3B models support context Windows of up to 128k tokens and are state-ofthe-art in their class or on device case uh use cases

Like summarization instruction following and rewriting tasks running locally on at the edge if you think about a lot of these Frontier models aren't even 128k or just are a little bit above 120k like uh 8K like 200k context window where whatnot so having an open- Source model like this where you can run it on your device and having that large of a context one is really gamechanging and I think this is really interesting you know speaking of the Llama stack and

I'm I'm very interested to see how this develops because they're really pushing this and I think it leads to like where metas and Zuck and you know their whole Grand Vision of this thing is and that's kind of their gameplay in this AI race is to get people to build on their open source tools and that's you know what Zuck has said so the Llama stack which will greatly simplify the way developers work with llama models in different environments including single

Node on Prem cloud and any device enabling TurnKey deployment of rag retrieval augmented generation and Tool enabling applications with integrated safety so it's pretty cool because I think it's going to this whole llama stack is going to bring maybe some more complex things that you would have to do with AI uh and make it a lot easier seamless more integrated and bring it to a wider um net of devs as well as you know obviously their whole reach with Instagram Facebook

You know them integrating within their apps closely the metaverse and now with their whole Oran glasses which I'll leave a clip to at the end of this video so make sure to stay tuned from that where zuck's announcing these if you haven't seen these These are pretty cool but integrating their open source Tools in these type of softwares is obviously going to bring AI to the Mas is in a way that a lot of other companies can't all right guys and

We can also see the vision instruction tuned benchmarks um for llama 11b and 90b comparative to Claude 3 ha coup and GPT 40 mini and we can see here on these scores like mmu mmu Pro um all these different ones right here you know text image right here for different uh tests it is really giving a lot of these top Frontier models obviously these aren't like claw son at 3.5 or anything like this but it's beating them in some categories and

Giving them a run for their money um and we're gonna like I said we're going to be doing some tests on uh these and I'll even do a couple little tests just you know just to see some of these Vision capabilities right now in today's video but we're going to be doing more extensive testing like we have for a lot of different models in future videos so make sure to stay tuned for that all right so here we can see an

Image understanding demo uploading an image now is analyzing this image of this fireplace and boom gives it a script ion modern open PL plan living space uh complement black leather Etc detect objects fireplace suggest Alternatives replacement uh suggestions so pretty cool stuff wow so then it's even you know generating alternative uh images based off that that's really cool all right and here they talk about how they actually created these lightweight and high performing models using pruning and distillation so pruning reduces

Model size by selectively removing parts of the networ while maintaining performance and distillation transfers Knowledge from larger models like llama 3.1 8B and 70b to smaller models and in post training they use alignment techniques and synthetic data generation to enhance capabilities like summarization and reasoning here we can see a mobile writing assistant demo right here so writing a sick letter to your boss and as it's generating this pretty quickly on the mobile device which is pretty impressive all right so you

Can go ahead and start using llama 3.2 today either on hugging face or you can go to AMA and I'll show you how to do this in just a second if you're not familiar with AMA and then you can also use it on sites like together AI as well as Gro okay Gro you can see right here uh we can access llama 3.2 the 90b text preview llama 3.23 uh 3B llama 3.2 1B and llama 11b if you don't know about

Gro it's essentially really quick inference times for llm models so I definitely recommend checking that out but I'm going to be showing you with together Ai and together AI is something I've talked about in other videos if you're not familiar with it you can go sign up for free I believe they give you a certain amount of free credits that you can try it out with I don't know if it's $5 or $10 or if they changed it at this point

But I remember when I lasted a video on it they uh were giving away some free credits but before we get started with together AI I just want to show you that if you're not familiar with olama basically go toa.com link will be down below you simply just sign up or download you can just click download actually and it's going to download the oama app on your desktop whether that's Mac Windows whatever the case may be once you do that you're

Just going to set up o llama and then it's very easy you can just come here to llama 3.2 which will be in the description you can select the model of your choice uh so the 1B or 3B and then you would just copy and paste this ol llama run llama 3.2 into your command terminal and it will set it up automatically then you'll have it locally on your device so the reason we're going to use together is because uh Gro

Doesn't have the um Vision capabilities right so uh with together AI they do have the 3.2 90b Vision instruct as well as the 11b vision instruct turbo so we're going to test out just a little bit of this just to see the image capabilities live in action all right so here we just have a very image of a very large data set of I don't even know what this is to be honest um and we can just see movement Deb J

Movement Deb J whatever the case may be I'm just going to give it these this data set and tell it please explain this large data set to me in a very easy to understand way explain to me what it is likely about and explain to me what the data tells you about this and what we can infer right so here you can see the data set appears to be a collection of numerical data organized into columns and rows each row represents

A single observation or measurement while each column represents a specific variable or feature being measured columns movement jet de J movement this column likely represents the type of movement or action being observed the values in this column are integers ranging from 1 to 51 suggesting that there are 51 different types of movement being studied Deb J okay so I'm not going to read through the whole thing but we can see of course that it did obviously read this and it did

Make some sort of uh inference based on the data that it's read and it's seems like it's pretty correct here so pretty cool stuff this is also something really cool I just realized about together AI you can see the UI version of what you just asked it but you can also switch to API and then you can switch to either typescript python or uh curl right here so we can see import OS import together uh import together or from together import

Together we can see the like we can literally see this this is actually really cool that it has this um typescript so yeah I think together a is really cool and if you're looking to build something with these new models I've always found that together AI is a good um platform to get access to these newer models because you're going to need to run them in the cloud if you guys know any other ones like together AI that does this and

It's pretty good I know there's some other ones let me know in the comments down below let me know what your favorite one is let me know if you guys have tried out this new llama 3.2 model whether that's the on edge models within your own devices hosting them locally whether that's in ama llm Studio whatever the case may be or if you've tried some of these out on the cloud or hey maybe you're already integrating some of these into your

Applications for you know right away for whatever reason so let me know down below what your experiences with these are already maybe if you haven't experienced them let me know what your thoughts are about this let me know what you guys think about zck strategy and meta strategy in this whole AI race and are you guys betting on some of these close Source models like Claud anthropic open AI I mean I don't know if you guys been following the AI news

But we're seeing a bunch of people leave open AI Mir moratti just left open Ai and a bunch of other top level people are leaving and have left you know so it's kind of interesting to see what's going on there I what do you guys think about that is it because of something behind the scenes is it because of Sam I don't know what do you guys think about that that's just a little side topic and then also too guys we

Seen at meta connect today Zuck announcing Orion and if you haven't seen that stay tuned to the end in just a second I'm going to put a clip of him basically showing Orion so you can see that and let me know what your thoughts are about that do you think that actually is going to um be really cool unlike maybe some of the older releases of the metaverse stuff even though it has started to get a little bit better um from

What we saw like it started to be a little bit cooler but it was kind of moving a bit slow for a while so do you think this new uh Orion thing is going to really take off and be something big let me know in the comments Down Below guys but anyways if you're new here we upload videos every single day on AI automation business growth Marketing sales Etc a bunch of different stuff if you like that type of content and

You got some value from this video make sure to smash that like button and subscribe to stay up to date with the daily uploads other than that guys stay tuned because like I mentioned I will be testing these models out and seeing how they compare to some of the other tests I did on 01 01 mini like 01 preview uh Claud Sonet 3.5 and let's see how some of these open-source models actually compare in these tests I'm very curious to see

And other than that guys I will see you in tomorrow's video stay tuned for that clip of Zuck if you haven't seen it already keep hustling keep grinding and of course guys accelerate your stride take care this is Orion our first fully functioning prototype and if I do say so the most advanced glasses the world has ever seen now about a decade ago I uh you I started putting together a team of the best people in the world to uh to

Build these glasses and the the requirements are actually pretty simple but the technical challenges to make them are insane um you know they they need to be glasses you know not a headset no wires less than 100 grams uh they need wide field of view holographic displays sharp enough to pick up details bright enough to see in different lighting conditions large enough to display a cinema screen or multiple monitors for working wherever you go whether you're you in a coffee shop

Or on a plane or wherever you are and you need to be able to see through them and people need to be able to see them through them too and make eye contact with you right this isn't pass through this is the physical world with holograms overlaid on it so if someone messages you uh you will see that and instead of having to pull out your phone there will just be a little hologram and with a few subtle gestures you can

Reply without getting pulled away from the moment or if you want to be with someone who is far away um they're going to be able to teleport as a hologram into your living room as if they're right there with you you're going to be able to tap your fingers and bring up a game of cards or chests or holographic pingpong or whatever it is that you want to do together you can work or play or whatever they're going to do voice

And AI they're going to do hand tracking and eye tracking so that way you can select UI Elements by looking at them but there's one more way that you're going to be able to interact with them that is really pretty neat a neural interface see voice is great but the thing is sometimes you're in public and you don't want to say what you're trying to do with your computer out loud hand tracking is neat for controlling different interfaces but you don't

Want to like walk down the street like this right so I think that you need a device that allows you to you just send a signal from your brain to the device so this isn't just the First full screen like you know full wide field of view holographic AR glasses this is also the first device that is powered by our wrist based neural interface [Music]