NEW Claude AI Agent Computer Use!🤖 Local Setup Tested! Claude Sonnet & Haiku 3.5 Controls Computer

What's going on guys Josh pook here and in today's video anthropic just released some crazy new models and features we got Claud Sonet 3.5 the new and improved version we got Claud Sonet 3.5 hi coup and we even got the new computer use AI agent where you can use the API and clog can control your computer it will take screenshots and make actions on your behalf to get things done we're going to go over some of the new model benchmarks and

I'm even going to show you how you can try out the new computer use feature in a virtual environment so you can test out this new AI agent capability yourself let's dive right into [Music] it all right guys so the main announcements like I said is the new version of Claude Sonet 3.5 the new and improved version and then we got Claud Sonet 3.5 high C so Claud Sonet 3 5 delivers across the board improvements over its predecessor with particularly significant

Gains in coding an area where it already led the field so this is really really cool uh I'm very excited to try this out more and I will be doing more tests with these new models in some of future videos so make sure to stay tuned for that so 3.5 ha coup matches the performance of Claude 3 Opus their previous largest model on many evaluations for the same cost and similar speed to the pre previous generation of hi coup this is

Great because it means better models cheaper cost and then they say we're also introducing a new ground biring capability in public beta computer use and I'm extremely excited for this because as you know on this channel we do a lot of different videos on different AI agent tools controlling your computer doing web searches and now we have one of the biggest Frontier Model companies who is actually releasing these type of features so it is currently available on the API today so

Now you can direct Claude to use your computer the way people do by looking at a screen moving a cursor clicking a button and typing text clonet 3.5 is the first Frontier AI model to offer computer use in public beta so it's still experimental at times cumbersome and eraone so this is quite interesting auna canva cognition door Dash repet and the browser company have already begun to explore these possibilities carrying out tasks that require doz and sometimes even hundreds of steps

To complete for example repet is using Claud Sonet 3.5 capabilities with computer use and UI navigation to develop a key feature that evaluates apps as they're being built for the repa AI agent project that's really interesting because we can see that these companies are already using similar things like these and this is the public beta that's being released to us I can only imagine what's going on behind the scenes at some of these bigger companies and how they're going to be

Using these models and how powerful that you know these agents are because they're essentially team members and you can build things so quickly so I'm extremely excited to see this start rolling out more and more now I'll leave links down below to the Claude official computer use videos right here where they show different tasks like automating operations and there's like two or three different videos so you can see in this example please fill out the vendor requests for ant equipment Co

Using data from their vendor spreadsheet or search portal tabs in window one list and verify each field as you complete the form in the window two and you can see that the tool is screenshotting things taking it step by step left clicking moving the mouse screenshotting all right as you can see it's actually doing those actions on the right hand side and it's filling in the form successfully and submitted it all right so like I said these videos will be linked

Down below all right so here's the benchmarks that they have and as you can see CLA sonit 3.5 the new version is pretty damn impressive it beats out the old claudet 3.5 in every single category pretty much and the 3.5 highq like I said before is now pretty much where the old Opus was and I'll leave a link down below to this blog post where you can read more about the Claude Sonet 3.5 update and the different Benchmark tests as well

As hi coup and then also two about teaching Claude to navigate computers responsibly you know on OS world and talking about the different benchmarks that they did for now let's actually dive into setting this up are you you tired of pouring thousands of dollars into appointment Setters only to watch leads slip away imagine having a team of elite sales agents booking qualified appointments for you around the clock no more wasted time on training no more frustration with performance and no more

Draining your budget on inconsistent and expensive call centers introducing stride agents AI powered appointment centers that work 24/7 never get tired and book appointments while you sleep trained on thousands of successful conversations our AI agents El performed human team at just oneth of the cost join the ranks of businesses that doubled their appointments and booking rates in just a matter of weeks don't get left behind in the AI Revolution visit stride agents.com now and transform your entire sales process with cuttingedge

AI technology it's time to accelerate your stride with AI agents okay so if we check out the API docs right here you can see a section which is called computer use beta and I'm not going to read through this entire thing and I'll leave it down below but you can see quad Sonet 3.5 the updated model is capable of interacting with tools that can manipulate a computer desktop environment we'll definitely test this out more too in future videos and setting it

Up with different AI agents with different tools and doing some more customizations so if you want to do that now I would suggest reading through these docs more in depth but for now we can see that they actually provide us a computer use reference implementation right here so if we click on this it's going to take us to the anthropic quick starts GitHub repo right here you can see we're in computer use right here so you can definitely read through some

Of this but it's pretty simple to get up and running so you can export your anthropic API key to your environment so you can simply just run this Command right here of course you're going to want to replace this with your API key okay so this is how you would set your anthropic environment variable if you're on Mac or Linux now if you're on Windows you're going to do it a little bit differently you can actually just run this Command right

Here and I'll leave it down below in the description if you are on Windows and same thing for getting this set up first things first you're going to need Docker installed so make sure you have Docker go to doer. set that up because this is going to be running in a containerized environment and then you're simply just going to run this right here in your command line and this is this is specifically for Mac or Linux if you're on Windows you're

Going to want to run this right here and like I said I'll leave both of these the windows version in the description okay so the windows version will be in the description and then if you're on Mac or Linux you can simply just follow this version okay you can see here once I ran this command it basically started computer use on Local Host 8080 now there are additional options if you want to use Bedrock or use vertex you can do so

And you can definitely change some of these too with the screen size for the different variables as well as here's the development version but we're just going to keep it simple and use the anthropic API quick start right here all right so this is what it's going to look like once you actually load up the app as you can see here we got a virtual environment right here using no VNC and then on the left hand side we have claud's computer

Used demo chat right here all right so we have two different views here we have the chat View and then the HTTP exchange logs View and once we actually start running some commands you'll be able to see both these populate now let's go ahead with our first one all right so our first prompt is search for Josh pook on YouTube and let me know how long his last YouTube video was how many views it has and how many subscribers he has

All right so we can see the agent is running now and we can see that it is going so every time it does a little action like that it makes that kind of noise we can see the tool use here computer input action screenshot same thing here Mouse is moving coordinate and by default toggle screen control is off which is kind of nice when the AI agent is doing things so you don't mess it up but if you did want to

Actually move things around on the screen you can simply toggle that on right so as you can see it's going through different screenshots it just opened up Firefox and now it's searching YouTube right here all right so we can see it searched up Josh pook right here all right so it just found my YouTube channel Josh pook okay so we got the results so looking at the search results I can see Josh pook's YouTube channel he has 9.22k subscribers thank you

For that by the way guys and his latest video titled free by data AI agent was uploaded 9 hours ago with 1.4k views and is 16 minutes and 4 seconds long to summarize boom boom boom okay so it got that correct all right so it took 15 different requests right here and we can go to http exchange logs to see all those if you click on this it will show you all the details between each request all right so we can

See the max tokens the message right here the system prompt the model all the different stuff the tools we're using all right next prompt we'll try is create a text file named pong. py on the desktop and fill it with some code on a pong game in Python then open the terminal and run it to play Pong all right so a little bit more complex let's see how it does oh my gosh okay so I don't know okay so I thought

It reached an error here so it was uh reaching a couple different errors here I thought it was I didn't think it worked and I guess it figured out a way to get around those errors itself I don't even know exactly what it did um okay so great now we have Python and the required dependencies installed and now let's run the python game and then it just ran it and as you can see there it was just playing it okay so

That's pretty cool that it actually created a pawn game and got it all set up and ran it um all by itself now let's try go to executiv st.com slapp and book a call for tomorrow at 400 p.m. using dummy data just fill in the form with whatever and answer the questions to book the call all right so let's go ahead and run this now trying uh filling out a form all right now I will say this is a bit slow

But this is pretty expected one it's in beta and two we're using a virtual environment in our browser right here so I wouldn't expect this to be lightning fast okay looks like we got an issue right here an error message so okay so I just went ahead and refreshed it so if it does glitch out ever just refresh the page and as you can see now it is running correctly okay so it entered in the website right here now it's going

To need to scroll down okay so it just messed up there all right so it kind of messed up if it does do the action before it takes the screenshot sometimes then it could kind of like Get lagged behind and then make the wrong move but it's going to end up ultimately oh no see now it's saying this okay so it's what it should be doing right now is scrolling down seems like it's having a bit of an issue with that

Wow okay so now now it just looks like it got the HTML content right here it's trying to find the calendar calendar is right here you just have to scroll okay so it basically said it couldn't do it um website might be using Advanced JS that's not loading properly in Firefox so yeah no it's incorrect here it simply just couldn't scroll down so I'm just going to I personally scroll down for it now let's see if it can I'm just going

To say book a call and see if it can book a a call here all right so I'm just going to say book a call on the calendar for the 23rd at 6:00 p.m. and just fill in dummy data so let's see if it can do this so I personally scroll down for it to see the calendar here so shouldn't be any excuse oh never mind it just reloaded okay it looks like it can see the calendar this time and it's

Clicking on the 600 p.m one now it looks like it closed Firefox okay so it's just been frozen here so I'm going to refresh it and try again all right so I ran the prompt one more time let's see if it can actually get it this time okay so I clicked on the 6 p.m now it should click on select by the way guys if you do want to book a call with our team this is how you do it I'll

Leave a link down below so if you need help growing your business then check this out okay so it filled in John now this is the issue I think it has an issue scrolling oh it just scrolled so maybe it's clicking enter or maybe it scrolled I don't know so I put John Smith there's quite a bit of data in this form though okay yeah so it did scroll okay that's good no oh it just changed the first name to an

Email and change the last name to a company and okay so it's filling it in incorrectly put the phone number in the job title position which is wrong okay I put bus okay it's completely messing up right now no now it's trying to click schedule event when it has complete wrong inaccurate information okay I don't know what's going on and okay this clicking is really starting to drive me crazy so I'm think I'm going to just say this as a fail

Epic fail I mean you could probably get this to work but I think we just with I mean it's very slow yeah no this is not going to work okay well other than that guys I think it's pretty cool I think uh you know the test demo that they provide like I said it is slow is not the best to use but it this is really just the start it's going to quickly pick up I can assure you of that and

Uh you know I'll be doing more videos surrounding this maybe using it in different agents but like I said you can go ahead test the demo right now try some different use cases let me know in the comments down below what you got this to do as well as what you think of this tool from anthropic this is pretty crazy along with those two new models you know it's really difficult to say who is leading the AI race right now between

Open AI anthropic we have other companies too we have Google this that whatever but we've seen some really cool releases recently from open AI now we're seeing some from anthropic and this is just really the beginning you know imagine a couple months down the line when this is a lot more advanced a lot more flushed out faster and imagine incorporating this with something like a real-time API or whatever the case may be maybe anthropic has their own version or whatever and

You just interact with your computer or your agent by simply talking to it and then it completely controls your computer maybe it completely controls your work or your development process or you know running your company and maybe you have a team of agents and they're doing all these different tasks and you're just orchestrating them or you have a manager that's orchestrating them an AI manager so a lot of cool stuff in the future um definitely excited to see and I think

This is just a stepping stone but let me know what your thoughts are in the comments down below other than that guys if you're new here we upload videos every single day on AI automation business growth so if you're new to the channel and you got some value here then make sure to like this video leave a comment down below and smash that subscribe button to stay up to date with the daily uploads you know tools like agent q and agent

E are definitely more optimal for browsing the web at this moment but like I said this is still the start and I definitely think that we will see some pretty cool ADV advancements with this very soon and also guys if you haven't already joined our free community stric community.com I'll leave a link down below our free Facebook group and Discord Channel also too guys if you run a business and you need help optimizing your systems booking more leads running paid ads

We're integrating AI agents such as AI cold callers AI appointment CS into your business and book a call down below executives.com apply and we can see if it's a good fit or not other than that guys I will see you in tomorrow's video keep hustling keep grinding and of course guys accelerate your stride take care