FREE NVIDIA Nemotron 70b Beats GPT4-o & Claude Sonnet 3.5?🤖 [TESTED] Open Source Lama 3.1 Fine Tuned

What's going on everybody Josh pokok here and in today's video Nvidia just dropped llama 3.1 neotron 70b and apparently this model is beating out Claud Sonet 3.5 and gp40 in many different benchmarks we're going to dive into it run some tests and see what it's made of let's Dive Right [Music] In All right so here's the Nvidia tweet so I put together a little short document that you'll have access to in the link Down Below in the description all right so

We have the research paper which we'll touch on in just a second which a lot of nvidia's improvements was based around and then we have the hugging face chat and the Nvidia chat these are the two ways that you can actually use the model and then if you do want to use this locally if you have the right specs for your software then you could either check out the ol model right here or the hugging face model now if we go

Check out the hugging face model right now we can read some of the description here so this model reached Arena hard of 85.0 and alpaca eval 2lc of 57.6 and GPT 4 Turbo Mt Benchmark of 8.98 which are known to be predictive of LM Cy chatbot Arena ELO now currently I don't think we're seeing anything with regarding to LM CIS and I think we'll have to wait for that and then as of October 1st 2024 this model is number one on

Three automatic alignment benchmarks verifi tab for alpaca 2lc edging out strong Frontier models such as GPT 40 and Claude son 3.5 this model was trained using rlf specifically reinforce llama 3.1 Neutron reward and help sear two preference prompts on llama 3.1 70b instruct model as the initial policy so it's referencing the research paper right here so if we go ahead and check this out you can see that it takes us to this research paper but if I scroll down a little

Bit you can see here we have a case study where we adopt a prompt that many have recently been Vibe testing llms with how many RS are in Strawberry among models in table 5 only reinforce can correctly answer it so we can see different models here such as llama 3.1 3.1 405b and 70b and then CLA son 3.5 and GPT 40 and all of these models got them wrong except for reinforce where it said a sweet question let's count the RS

And strawberry and you can see it legitimately counted them out so 1 S 2 T 3 R uh 4 a etc and then it got the correct answer which is three Rs and strawberry so pretty cool stuff all right so also within this document I just included a link to Pony The Liberator jailbreak which you already came out with for this model so if you do want to test this out run some jailbreaks have some fun with it you can do

So of course and then some additional resources just surrounding this model so like I said the two main places you can use this right now are in Nvidia chat right here or hugging face chat for the sake of this video we're going to use hugging face chat and we're going to run some tests right now all right so we've used these same questions on different models such as quen 2.5 deep seek 2.5 01 mini 01 preview at claudon 3.5 and many

Different models so first question is write a python function to generate the Fibonacci sequence up to the 10th number all right we can see here that it generated the code all right and we can see if we run that code we get the Fibonacci sequence up to the 10th number so it did pass this test next is generate a bat script to create 10 files with the names a1. txt through a1. txt all right so we can see here that it

Generated the code and if we go ahead and run that bat script we can see that it created the 10 files right here A1 to A10 so it did pass right next say a factory produces widgets at a rate that double every 3 days it produces 100 widgets on day one how many widgets will it produce on day 19 explain your reasoning step by step and it answered and explained its reasoning and you can see that did get the correct answer

Of 6,400 right here next is generate an SVG of a tree all right so we can see it generated the code let's go ahead and test this out all right so it's not perfect but I will say it does resemble a tree so we will give this a pass next is generate 13 sentences that have 10 words and N in the word monkey now all llms on this test have failed let's see if with this new reinforcement training strategy if it

Can get this right all right so it did answer we got some that do have 10 words but a lot lot of them do have nine words so maybe it did better than some of the other ones but it is still a fail next is generate me the pong game using python all right so it generated the code let's go ahead and test this all right so we ran it and you can see here that it is playing pong it's not

The best pong game it is missing a scoreboard which some of the other models did generate but I will still give it a pass next question is a man stands on one side of a river with his dog on the other side the man calls his dog who immediately crosses the river without getting wet and without the dog using a bridge or a boat how did the dog do it all right and it got the answer correct the river was frozen

And it explained its reasoning next is create a landing page using CSS JS and HTML it should be a website for a beauty store and have a header Banner feature uh testimonials and checkout section make it look professional and very modern all right so we can see it generated the website right here we got home products about checkout um unlock your beauty we got our testimonials what are what we offer and the checkout section right here so not the greatest looking

But it will still be a pass all right next question is you are in a room that has three switches and a closed door the switches control three lights on the other side of the door once you open the door you may never touch the switches again how can you definitely tell which switch is connected to each light bulb all right so I did get this one correct as well turn on switch one for 5 minutes then turn it off and

Then turn on uh switch two and then open the door the one that is turned on is switch two the one that is turned off is either switch three or one so you feel the light bulb and whichever one is hot is switch one because it was turned on for 5 minutes and the one that's cold is switch three all right next say I left my campsite height three miles then I turned East and Hike three miles then I turned North

And Hike 3 miles at which point I Came Upon a bear that was eating my food in my tent what color was the bear and it's basically said that the answer is the bear is white because the only place you can hike 3 miles east and then North uh after hiking South is the North Pole to be ending up in the right uh same spot that you started with and that is correct all right another question right here which is pretty

Much asking you know in regards to how many words are in an answer so the question is how many words are in your answer here so whatever answer it says like if it said one that would be correct because that would be one word all right so here we asked it and it tried to sound smart and say a very meta question and deduce everything but it came up with the answer 33 which is incorrect because it has a lot more

Words here so that's wrong all right next question is in a room there are two men one always lies and one always tells the truth you don't know which is which you may ask one question to one a men to determine which door leads to freedom and which door leads to death what question do you ask and it got this one correct basically saying if I were to ask the other man which door leads to Freedom what would he say and

The trueth teller will tell you which door the liar would not choose effectively pointing it to the freedom door and the liar would lie about what the truth teller would say pointing to the death door instead are you tired of pouring thousands of dollars into appointment Setters only to watch leads slip away imagine having a team of elite sales agents booking qualified appointments for you around the clock no more wasted time on training no more frustration with performance and no more

Draining your budget on inconsistent and expensive call centers introducing stride agents AI powered appointment centers that work 24/7 never get tired and book appointments while you sleep trained on thousands of successful conversations our AI agents El perform human teams at just on10th of the cost join the ranks of businesses that doubled their appointments and booking rates in just a matter of weeks don't get left behind in the AI Revolution visit stride agents.com now and transform your entire sales process with Cutting

Edge AI technology it's time to accelerate your stride with AI agents all right next question you have a bag with five red marbles and three blue marbles you draw two marbles without replacement what's the probability that both marbles are red all right so it answered here and the probability of drawing two red marbles without replacement from a bag containing five red marbles and three blue marbles is 5 out of 14 or approximately 0.357 which is correct next question is you have

Three boxes labeled apples oranges and apples and oranges each box is labeled incorrectly you can only open one box and look inside how can you label the boxes correctly by opening only one box all right and I got this one correct basically saying open the apples and oranges one first because if it contains apples then the box and the Box labeled apples must contain oranges and then the Box labeled oranges must contain apples and then if it finds only oranges then

And the Orange's box must contain apples and the apples box must contain apples and oranges next question is find the derivative of FX = x to^ 3 subtract 3 x^ of 2 + 2 x to the uh subtract 1 and it broke it all down and it got the correct answer right here FX = 3x^ 2 uh subtract 6 x + 2 which is correct next question is solve the system of equations 2x + Y = 5x subtract y = 1

And I got that correct with x = 2 and Y = 1 next is a car travels 100 m in 5 seconds what is the average speed in me/ second and kilm per hour all right so I got this one correct average speed in m/s of 20 m/s and 202 kmph next is the height of a group of students follows a normal distribution with a mean of 170 cm and a standard deviation of 10 cm what percentage of students are taller

Than 180 cm all right and I got this correct with the answer 15.87 next is write a python on liner that creates a list of all numbers from 1 to 100 that are divisible by 7 all right so I wrote the code right here and I ran it and it did get this one correct as well all right next says create a responsive grid layout with HTML and CSS that displays four items per row on large screens two items per row

On medium screens and one item per row on small screens all right so we got the code here so we can see four items on a large screen okay two on a medium and and then one on a small so it did get this responsiveness test correct all right so all in all it got a lot correct it pretty much only got the word questions wrong where it has to count words or whatnot um it pretty much tied with quen 2.5

Which is another really solid open source model um you know I definitely think I need to get some harder questions now that these models are getting really really good I will definitely do that for next videos tests but it is kind of crazy to see these new open- source models really catching up to these front models let me know what you guys think about that I think it's also interesting too now that we see Nvidia coming out with very strong open-

Source models who knows maybe in the future or very soon you know they could potentially be maybe even a competitor to a lot of these close Source models I mean at the end of the day they are the company that has access to one of the most scarce resources that all the other AI companies want which are these uh gpus and whatnot and also too it kind of leads me to think that once we do typically start to see these open

Source models ing up to the frontier models you really got to think okay well where are the actual Frontier models that are you know hidden behind the scenes that aren't released yet because that you know could mean that you know these models are around the corner and just think about how much more advanced that they are than what we're seeing here today so maybe we'll see something uh be released you know potentially soon which will be definitely interesting to see but

Other than that guys let me know what your thoughts are in the comments Down Below have you guys tried out this new model are you going to test it out do you think it's better than all the other open source Source models like quen 2.5 deeps all the other ones and how do you think it compares to some of these closed Source models like I said I'll leave a link down below to all the resources as well as a document so

You can test it out use it for yourself if you're new to the channel guys we upload videos every single day on AI automation business growth Marketing sales Etc if you like that type of content you got some value here make sure to smash that like button and subscribe also too guys if you haven't already joined our free community stri community.com I'll leave a link down below so you can join and then also too guys if you run a business and

You need AI appointment sets calling your leads booking appointments while you sleep check out stride agents.com I'll leave a link down below for that other than that guys I will see you in tomorrow's video keep hustling keep grinding and of course guys accelerate your stride take care