BREAKING: OpenAI Unveils GPT-4o - FREE, FASTER, MULTIMODAL - SHOCKED The ENTIRE World! 🤖🌎

Breaking news chat GPT 40 just was released we're going to go through the in-depth overview of what this actually means what the announcement was let's dive right into [Music] it all right guys so May 10th Sam Alman posted on Twitter saying not GPT 5 not a search engine but we've been hard at work on some new stuff we think people will love feels like magic to me Monday 10:00 a.m. Pacific Standard time and you know people were speculating what could this

Be now we officially got our answer it is gp40 and we're going to go through the actual video and I don't want to spoil too much um of what this actually entails and what what the features are within gbt 40 but it is pretty pretty crazy just to give you some context here was a tweet saying GPT 40 is our new state-ofthe-art Frontier Model we've been testing a version on the LM CIS Arena as I'm also as I'm also a good

Gpt2 chatbot which people were kind of speculating what this gpt2 chatbot could be and here's how it's been doing so you can see here overall e ELO score right and you can see here um GPT 40 AKA GPT 2 is clearly ahead of gp4 Turbo GPT 4 Gemini and all the other models so let's see what this actually entails with this announcement hi everyone Hi everyone thank you thank you it's great to have you here today today I'm going to talk

About three things that's it we will start with why it's so important to us to have a product that we can make freely available and broadly available to everyone and we're always trying to find out ways to reduce friction so everyone can use chig wherever they are so today we'll be releasing the desktop version of chbt and the refresh UI that makes it simpler to use much more natural as well but the big news today is that we are launching our

New flagship model and we are calling it gbt 40 the special thing about gb40 is that it brings gb4 level intelligence to everyone including our free users we'll be showing some live demos today to show the full extent of the capabilities of of our new model and we'll be rolling them out iteratively over the next few weeks all right so let's get started a very important part of our mission is to be able to make our Advanced AI tools available to

Everyone for free we think it's very very important that people have an intuitive feel for what the technology can do and so we really want to pair it with this broader standing and we're always finding ways to reduce that friction and recently we made CH gbt available without the signup flow and today we're also bringing the desktop app to Chad gbt because we want you to be able to use it wherever you are as you can see it's easy it's simple

It integrates very very easily in your workflow along with it we have also refreshed the UI we know that these models get more and more complex but we want the experience of interaction to actually become more natural easy and for you not to focus on the UI at all but just focus on the collaboration which had gbt and now the big news today we are releasing our newest Flagship model this is GPT 40 [Applause] gp40 provides gp4 level intelligence but it

Is much faster and it improves on its capabilities across text vision and audio for the past couple of years we've been very focused on improving the intelligence of these models and they've gotten pretty good but this is the first time that we are really making making a huge step forward when it comes to the E of use and this is incredibly important because we're looking at the future of interaction between ourselves and the machines and we think that GPT 40 is

Really shifting that Paradigm into the future of collaboration where this interaction becomes much more natural and Far Far easier but you know making this happen is actually quite comp Lex because when we interact with one another there's a lot of stuff that we take for granted you know the ease of our dialogue when we interrupt one another the background noises the multiple voices in a conversation or you know understanding the tone of voice all of these things are actually quite complex

For for these models and until now with voice mode we had three models that come together to to deliver this experience you have transcription intelligence and then text to speech all comes together in orchestration to deliver voice mode this also brings a lot of latency to the experience and it really breaks that immersion in the collaboration which had GPT but now with GPT 40 this all happens natively gbd4 reasons across voice text and vision and with this incredible efficiencies it also

Allows us to bring the GPT 4 class intelligence to our free users this is something that we've been trying to do for many many months and we're very very excited to finally bring gbt 40 to all of our users today we have 100 million people more than 100 million in fact they use CH GPT to create work learn and we have this Advanced tools that are only available to our paid paid users at least until now with the efficiencies of 40

We can bring these tools to everyone so starting today you can use gpts and the GPT store so far we've had more than a million users create amazing experiences with gpts these are custom chat gpts for specific use cases they're available in the store and now our Builders have a much bigger audience where you know University professors can create content for their students or podcasters can create content for their listeners and you can also use Vision so now you can upload

Um screenshots photos documents containing both text and images and you can start conversations with chat gbt about all of this content you can also use memory where it makes CH GPT far more useful and helpful because now it has a sense of continuity across of all your conversations and you can use browse where you can search for real- time information in your conversation and Advanced Data analysis where you can upload charts or any information and it will analyze this information it

Will give you answers and so on lastly we have also improved on the quality and speed in 50 different languages for chb and this is very very important because we want to be able to bring this experience to as many people out there as possible so we're very very excited to bring d 40 to all of our free users out there and for the paid users they will continue to have up to five times the capacity limits of our free users

So that's pretty crazy just to give a brief overview you know just to cover what we just heard we heard we're getting a GPT uh desktop app which is pretty cool I already had one that I just got off GitHub but it's cool to finally have a native desktop app second is GPT 40 um being available to free users which is really interesting so like she said in the video they already made GPT um chat GPT available without having to be

Signed in so I'm in an incognito window you don't have to sign up anymore you could literally use gpt3 without 3.5 without signing in um I don't know exactly if they did this update after meta or before I'm assuming if it was after I mean it was obviously because meta this out so if we go to meta doai you can see here right here we can go ahead and start a chat with meta continue without log in boom but yeah so

I would assume that they did that because of that and now she's mentioning gbt 40 available for free users which is really insane um and then the paid users just have five times the Capac capacity I wonder if there'll be any other specific features uh for the pro users I'm assuming there must be something if they're making GPT 40 available to free users probably going to have some additional stuff for paid users or some additional rollouts coming soon who knows maybe

Sora maybe whatever the case may be I saw some people on Twitter like well why am I paying for GPT 40 Pro if everyone's going to get it but obviously the Capa extra capacity is good but let's see um within the next few weeks what else uh it will inail in the pro pro version and then O is for omnipresent I believe it says it on the screen there so she's going to get into that in a second uh and kind

Of see what that actually entails for GPT 40 but gbt 40 is not only available in Chad GPT we're also bringing it to the API so so our developers can start building today with GPD 40 and making amazing AI applications deploying them at scale 40 is available at 2x faster 50% cheaper and five times higher rate limits compared to gbd4 Turbo that's pretty crazy having uh two times faster 50% cheaper and five times higher rate limits than GPT for Turbo cuz

Turbo is was already the best model now GPT 40 is way faster and way cheaper that's going to be insane for agents but you know as we bring these Technologies into the world it's quite challenging to figure out how to do so in a way that's both useful and also safe and gbd 40 presents new challenges for us when it comes to safety because we're dealing with realtime audio realtime vision and our team has been hard at work figuring out how

To build in mitigations against misuse we continue to work with different stakeholders out there from government media entertainment all Industries red teamers Civil Society to figure out how to best bring these Technologies into the world so over the next few weeks we'll continue our iterative deployment to bring up all the capabilities to you but today I want to show you all these capabilities so we'll do some live demos I will bring on two of our research leads Mark Shen and Barett

Z to join me hi I'm uh I'm Barrett hey I'm Mark so one of the key capabilities we're really excited to share with you today is realtime conversational speech let's just get a demo fired up so I'm taking out a phone if you are wondering about this wi so we have consistent internet and if you see there's this little icon on the bottom right of the track GPT app and this will open up GPT 40's audio capabilities hey chat GPT I'm

Mark how are you oh Mark I'm doing great thanks for asking how about you hey so I'm on stage right now I'm doing a live demo and frankly I'm feeling a little bit nervous can you help me calm my nerves a little bit oh you're doing a live demo right now that's awesome just take a deep breath and remember you're the expert I like that suggestion let me try a couple deep breaths can you give me feedback on my breaths okay

Here I go whoa slow a bit there mark you're not a vacuum cleaner breathe and account of so you can hear the demos kind of cutting out in and in and out here um so I mean you know it's only going to get better but we're going to see more of this but it already it's insane it's a huge improvement from the previous voice talking the The Voice just how it sounds how authentic it sounds it sounds better than most humans

Voices and um also uh it's way faster like this like for example we use softwares for our clients and our businesses for like call calling agents like AI calling agents and the the latency and the speed was so slow in terms of it just wasn't there yet I mean it still could work it still was working but in terms of like actually being able to cold call someone or just appointment set someone with an AI voice and have it sound authentic

It just was a bit slow if you asked a question this is going to be gamechanging for business for everything and it's only going to get crazier like I just wait till we get to some of these features here for okay uh let me try again so I'm going to breathe in deeply and then breathe out for four and then exhale slowly okay I'll try again breathing in and breathe out that's it how do you feel I feel a lot better

Thank you so much so Mark you've been working on these capabilities for a while now can you tell us a bit how it's different from voice mode right so if you've used our voice mode experience before you'll notice a couple key differences first you know you can now interrupt the model you don't have to wait for it to finish your turn before you can start speaking and you know you can just butt in whenever you want second the model is Real

Time responsive so that means that you don't have this awkward 2 to 3 second lag before you wait for the model to give a response and finally the model picks up on emotion right when I was breathing super hard there it could tell and it knew hey you might want to calm down a little bit you're kind of running a little bit too fast so you know it really has that capability across the board to perceive your emotion not only that

Though the model is able to generate voice in a variety of different emotive Styles and it really has a wide dynamic range so I'm going to fire up another demo for that yeah hey chubbt how are you doing I'm doing fantastic thanks for asking how about you pretty good what's up so my friend um Barrett here he's been having trouble sleeping lately and uh I want you to tell him a bedtime story about robots and love oh a bedtime story about

Robots and love I got you covered gather round Barrett Once Upon a Time in a world not too different from ours there was a robot named bite bite was a curious robot always explor started the story but I want a little bit more emotion in your voice a little bit more drama got it let's add some drama once upon a time in a world not too different from ours there was a robot named no nobt I really want maximal emotion like

Maximal expressiveness much more than you were doing before understood let's amplify the draw drama once upon a time in a world not too different from our yeah so that's crazy with the inflection and the tonality and the all that stuff that I can pick up with emotion and stuff that is just insane better than most humans um now does it s like that's the thing it's a so it's better than most humans it sounds a lot more realistic um you know

It's still you could probably tell that it's you know an AI if you're listen if you kind of know about AI but the average person that may not know about AI that much um may actually just believe it's a real person talking just with a really good voice right and I'm sure they have different voice um different voices this is just as female voice but they definitely have different options so um you can change the voice sounding and it'd be interesting

To hear these other ones as well but that's insane how quick and how um how high quality this is our there was a robot named bite bite can you do this in a robotic voice now initiating dramatic robotic voice once upon a time in a world not too different from ours there was a robot named bite bite was a curious robot always exploring new circuit see Barrett here calm down a little bit can you end the story um but do it

In a singing voice and so robit found another robot friend and they live circly Ever After thank you so much I definitely feel a lot more relaxed youve just seen the voice capabilities but we also want to show you the vision capabilities as the model can also see the whole world around us yeah absolutely like as you can see you know you can chat with chat gbt in real time but you can also interact with it um with video as well

Okay let me boot up chat PT hey chat CHT hello there how's it going it's going really well today I'd really like your help solving a math problem I'm all ears what math problem can I help you tackle today so I'm going to write down a linear equation on a sheet of paper and I'll show you and then I love your help working me through it but importantly don't tell me the solution just help give me hints along the way got

It okay I see it no I didn't show you yet just give me help along the way one second whoops I got too excited I'm ready when you are so Chachi PT what equation did I write down there ah ah I see it now you wrote down 3x + 1 = 4 yep exactly so what's the first step I should take to try to solve this the first step is to get all the terms with X on one side and the

Constants on the other side so what do you think we should do with that plus one okay I'm going to try to subtract one from both sides and then I'll see what I get great idea perfect now what do you get when you subtract one from both sides okay so chat GPT this is what I ended up with how does this look spot on now you've isolated the term with X on one side and it looks like 3x equal three what

Do you think we should do next to solve for x I'm not sure can you give me another hint of course you have 3x and you want to find the value of x think about what operation undo multiplication is it subtraction close but think of the opposite of multiplication because 3x means 3 * X okay I think it's division let me try to divide both sides by three and let me see what I get Bingo go ahead and divide both sides

By three so it's pretty much like better than or just as good as like a tutor right this is barly simple equation but you know even on more complex equations or just anything where you have to like it's like a a full out assistant a helper that can see the actual world and help you along the way so I'm just thinking about all the possibilities that you could use it for and uh we're going to get into that in just a

Second but it's really insane okay so chat gbt this is what I ended up with how does this look it looks perfect you've solved it and x equals one nicely done how do you feel about solving linear equations now yeah honestly i' I'm not pretty confident I can solve linear equations but why would you ever need to use this in the real world I'm really skeptical that's a fair question linear equations pop up in a lot of everyday situations even if

We don't realize it for example calculating expenses planning travel cooking and even in business for profit and loss calculations it's basically a way to solve problems we you need to find an unknown Value Plus wow you know I have a lot of unknown values in my life I'm totally convinced we I'm gonna learn math now I love that attitude Matt can definitely help solve some of life's mysteries whenever you're ready to dive into more math I'm here for you anything else

You'd like to tackle today so chbt I really love that you you know taught the value of math to my friend Mark and I wrote one last thing I'd love if you could take a look at of course I'd love to see what you wrote show it to me whenever you're ready okay so this is what I wrote down what do you see a I see I love chat CH PT that's so sweet of you yeah well I really appreciate all

The help so yeah as we can see yeah wow that's quite the outfit you've got on yeah as we can see you know we can chat in real time to chat GPT and not only that we can interact with it with video as well and I swear like if you look at Barrett's GPT history it's all equations like this now that you know you can do linear equations can you show us something more complex I think so okay next let's try

Something uh coding related so let me take out um a computer for this okay so I'm going to open the chaty BT desktop app like miror was talking about before okay and to give a bit of background of what's going on so here we have um a computer and on the screen we have some code and then the chat gbt voice app is on the right so chat gbt will be able to hear me but it can't see anything on the

Screen so I'm going to highlight the code command C it and then that will send it to chat GPT and then I'm going to talk about the code to chat GPT hey chat GPT hey there how's it going yeah it's going really well I was wondering if you could help me with a coding problem today of course I'd love to help you out what's the coding problem you're dealing with okay I'm going to share with you some code one second sure

Thing take your time okay so I just shared some code with you could you give me a really brief one- sentence description of what's going on in the and I'll let this play out but just think about like this is what I'm thinking like oh crap now we have this desktop app with a little voice box where you're literally going to be able to be working on your computer and have the chat GPT assistant in the corner or wherever and you

Can start just talking to it and you can we we'll get into it in a second but you can literally show show it stuff on your screen and all this stuff so now it's a much easier interface than just having to pull up the the chat and start typing to it and you know you have to ask it this question you have to type it out whatever the case may be um I liked it on the phone when you can use

The voice I used to always like talk to it on the phone but now to be able to show at your screen or copy and paste things over to the screen and ask it about your work or whatever you're doing that is gamechanging for developers or business owners for really anyone so let's see how this works the code this code fetches daily weather data for a specific location and time period Smooths the temperature data using a rolling average annotates a significant

Weather event on the resulting plot and then dis displays the plot with the average minimum and maximum temperatures over the year awesome thanks so much and in the code there's this function Fu could you describe to me what the plot would look like if I had versus didn't have this function sure the function Fu XY takes in a data frame X containing temperature data and a window size y for calculating the rolling mean it Smooths the average temperature minimum temperature and

Maximum temperature columns by applying a rolling mean over the specified window with the foo function the plot would show smoother temperature lines because the rolling mean reduces the noise or fluctuations in the data awesome this makes thanks so much so yeah I definitely will want to apply this function and so I'm going to uh generate the plot and then show it to you give me one second absolutely take your time I'm excited to see the plot when you're ready yeah so

So far chat gbt has only been able to see the code and now I'm going to run it and then I'll use the vision capabilities of the chat PT desktop app so it can actually see everything going on on the screen as well okay chat PT I'm sharing with you the plot now I'm wondering if you can give me a really brief one- sentence overview of what you see plot displays smoothed average minimum and maximum temperatures throughout 2018 with a notable

Annotation marking a big rainfall event in late late September fantastic I have another question for you um which months do you see the hottest temperatures and roughly what temperatures do those correspond to the hottest temperatures occur around July and August the maximum temperature during these months is roughly between 25° and 30° 77° F to 86° F what's going on with the y axis is this in Celsius or Fahrenheit the Y AIS is in Celsius the temperatures are labeled accordingly such as

Average temperature minimum temperature and maximum temperature yeah so as we can see you know not only can trat PT help me solve very easy linear um algebra equations that I need help with it can also interact with you know code bases and see the outputs of plots and everything like this going on on a computer so a few minutes ago we asked the live audience on X to submit few requests for what they would like us to try out here so

I will take a couple of prompts okay bot gasar wants to know if gbd4 o is capable of realtime translation Mark you want to try this one sure yeah let's do it I speak Italian so we can just English Italian try to do English Italian sure let's do it hey Chachi how are you I'm doing fantastic thanks for asking how about you how's your day going I'm uh doing great so I would like you to function as a translator I have

A friend here who only speaks Italian and I only speak English and uh every time you hear English I want you to translate it to Italian and if you hear Italian I want you to translate it back to English is that good perfo Mike she wonders if whales could talk what would they tell us um they might ask uh how do we solve linear equations certainly yes great looks like it works all right so the next one is from burrito John

78 who asks can you tell what you're feeling just by looking at your face Barrett you want to give this one a try absolutely let's try it out hey chat GPT hey there what's up how can I brighten your day today okay yeah so I'm going to show you um a selfie of what I look like and then I'd like you to try to see what emotions I'm feeling based on how I'm looking sounds like a fun challenge go ahead and

Show me that selfie and I'll put my emotional detective hat on okay so here's me so what kind of emotions you think I'm feeling H it seems like I'm looking at a picture of a wooden surface oh you know what that was the thing I sent you before don't worry I'm not actually a table um okay so so take a take another look uh that makes more sense a there we go it looks like you're feeling pretty happy and Che f

With a big smile and maybe even a touch of excitement whatever's going on it seems like you're in a great mood care to share the source of those Good Vibes yeah no uh the reason I'm in a really good mood is because we were doing a presentation showcasing how useful and amazing you are oh stop it you're making me [Applause] BL amazing well that's it for today on live demos as you can see this is this just feels so magical and

And that's wonderful but we also want to remove some of the mysticism from the technology and bring it to you so you can try it for yourself so over the next few weeks we'll be rolling out these capabilities to everyone today has been very much focused on the free users and the new modalities and new products but we also care a lot about the next all right guys so that's pretty much it for GPT 40 roll out this is huge news

This is a major win for open AI in my opinion you know the AI scene has kind of been moving a bit slow I mean there's been a lot of news but in terms of actual progression um you know we've had things like anthropic we've had things like you know Gemini we've had you know Gro and all these different updates but they have really just been trying to play catchup to open AI so now we're actually seeing open AI the front

Runner actually push the AI scene forward push the boundaries of what we've actually been able to seen here with GPD 40 omnipresent modalities um I think this is going to be insane for what you can do for your business for what you can do for if you're a coder a developer if really anyone I think now that they're actually applying this to the free version is huge because so many people that aren't in the AI scene like students I mean people

Already are using GPT but now that you'll be able to use these GPT features for free it's going to be insane I think it's going to really expand the GPT uh open AI you know users user um user base as well as um so many people are going to be uh now that they're actually going to be getting all this real world data from different images and all the multim modalities is going to really increase GPT fors or GP open a

Eyes training data um they're going to get so much data so it's going to make the models even more powerful now if you do want to use GPT 40 right now like like uh M morat said in the video they'll be rolling it out to users in the next couple weeks but if you go to platform. open.com um you can use the playground chat assistance I don't know about the assistants yet but you can use a playground and the API keys

For gp4 right now so you can go ahead I'll be making more videos on GPT 40 and actually using it I personally for using a tool called team GPT which allows you to pull your API key and use GPT 40 in a similar way to you would with GP uh chat GPT and it's really really cool so I use that um you can use any API key pretty much uh claw Gemini Etc but uh other than that I'm doing a video

A day right now so uh make sure to smash that subscribe button let me know what type of videos you guys want to see let me know what your thoughts are in terms of gp4 roll GPT 40 roll out what you think about this whole situation I'm going to be making a lot more AI application videos in the future in the next few weeks so how to actually use gp40 in your business in your personal life build different automations different agents

Etc so make sure to subscribe and I will see you in tomorrow's video guys keep hustling keep grinding and of course accelerate your stride take care guys