FREE: Agent-E AI Browser Agent Controls EVERYTHING!🤖 AutoGen Open Source AI Framework Emergence AI

What's going on everybody joshh pook here and in today's video we are going to be looking at agent e an AI agent system that can literally control your computer through your browser this is a browser automation framework that is based on autogen and it is extremely powerful and really accurate I'm going to show you how you can get it set up and start using it in under five minutes step by step let's dive right into it [Music] all right guys so

Three days ago I actually made a video on agent q and if you didn't see that it's essentially an AI agent that can do something that's pretty similar to agent e right so it can basically control your browser um but it works a little bit differently and we're going to dive into that right now so agent e is an agent-based system that aims to automate actions on the user's computer at the moment it focuses on automation within the browser the system

Is based on autogen agent framework this provides a natural language way of interacting with a web browser fill out forms search and sort products locate specific content navigate to and interact with web-based media perform comprehensive web searches manage and automate tasks on product management platforms provide shopping assistance you know a bunch of different stuff while agent e is growing it is already equipped to handle versatile range of tasks but the best task is the one that you come up with so

Take it for a spin and see what it's all about so there's a Blog article right here we'll touch on that in just a second and then it goes through the instructions now here is the blog article distilling the web for multi-agent automation okay so it goes through kind of the whole um you know framework of agent e showing some different examples and videos here some different diagrams and you know it performing different tasks and also too they have a research

Paper so agent e right here and this is the research paper um I'll leave a link to all this in the description down below I also put together a little document right here which is kind of like a resource for pretty much anything you would want to know regarding agent e so I did the same thing for agent Q so you can kind of compare and contrast a post from agent Q versus Agent e all right so first things first we

Obviously have the GitHub repo we have the agent e blog post um I also included the auto genen docks because if you're going to be doing more custom things with agent e um like you can do some more advanced things you may want to kind of reference the AG autogen dogs which is the the framework it's based off now the research papers are right here so we got the agent e research paper we also got two other research papers that agent

E references so the autogen research paper here and the web Arena research paper and then we got agent Q alternative as well as the doc right here and that also too it's kind of similar to something like open interpreter so we got some additional information on agent e here the main things I want to kind of just quickly go over is the architecture so it's based on agent oriented programming so aop and then there's four key characteristics so sensing processing action

And self-improvement and then hierarchical architecture with separate planning and browser navigation agents and then skills SL Atomic action so so divided into sensing skills so get URL get Dom with content type and action skills such as click enter text and then skills provide natural language feedback for better error handling and then Dom distillation so this is covered in the blog uh and they also have a short video on it but compacts large HTML Doms to fit within LM llm context Windows

Reduces tokens needed to represent web pages and creates hierarchical Json representation of relevant Dom elements all right so these are a few key things and I personally feel like I noticed them a little bit just using agent e and we'll get into that in just a second it feels really nice in terms of using it and how quick it is at uh performing tasks next is planning and reasoning so llm prepares action plans based on user instructions and then can adapt

Plans or request clarification when faced with ambiguities and then extensible architecture so easily add new skills using the auto gam framework so that's pretty cool very customizable that's why included the autogen docks and this is a tool that I actually am going to be playing around with more and trying to experiment with this because I think browser-based um interaction with AI agents is really the future here and then considering lightweight router for compact skill representation and then the key components is

Intelligent Dom distillation like we talked about scalable skills registry uh architecture okay and I did include a little diagram here of agent Q versus Agent e so not going to go over the whole Entre ire thing if you want to check out my agent Q video you'll get a sense for it and you'll really understand one of the main things I do want to mention though that we haven't really touched on yet uh is the performance so 50.5% on web shop

And 95.4% on Open Table is some uh tests that agent Q got and then agent e got 73.1% overall on web Voyager Benchmark okay so if you want to read through some of that additional stuff you can check that out in the doc below it's down there for free so you can reference this um now a few prerequisites you're going to want of course before we get into the install is git Python and then the open AI API key now you

Can use this with pretty much any model so I'm just going to be showing you with open AI but you could use this with Gro you could use this with Gemini whatever the whatever you uh desire so first things first is you're going to want to get clone the repo here so I already did that you can see here I ran get clone I ran get clone right here and that's going to clone a agent e and then you're going to

Want to CD into agent d e all right so the commands are all right here in this document they're also in the GitHub repo so there is a couple like a couple ones that um I just added in here just for clarity for you guys but you can either reference the GitHub repo or this document either one works then you're going to want to install UV so you can either install UV by just running this script right or this line right

Here this is for Mac um right here actually and then so yeah for mac and or Linux you would just run this script for Windows you would just run this script now alternatively you could use uh pip by just doing pip install UV so you have those different options there once you do that that's going to install UV and then you're going to want to use UV to create and activate a virtual environment for the project so so you can run

This line right here UV and then space ven uh v n V and then D- python 3.11 3.10 plus that should work also they say and then if you're on Mac or Linux you can do Source this line right here to activate the virtual environment on Windows you're going to run this line right here once the virtual environment is activated you're going to want to install the dependencies okay so to do so you're just going to uh run UV pip compile

P project. tl- requirements.txt okay that's going to start installing those uh requirements here and you're also going to run UV pip install requirements.txt that's going to start uh installing the requirements try you could install these extras right here so UV pipe uh pip install dasr projects. toml and then D- extra Dev this is the same as the requirements here this is just has the extra Dev flag on it so um I don't think it's required but you know you could do

Either one and then this is optional so you can install playright drivers I did this just because you know whatever um so if you don't have Google Chrome installed locally and you don't want to use it then or you don't want to install it then you can use playright for browser automation so just play right install and then you're going to configure the environment so you're going to create a EnV file by copying the provided example file so you can do

CP copy uh the environment example Le and then EnV so just run this command um and then for me I actually after this I just did code dot just because it's easier to for me to do this in in vs code right so that's going to open it up in vs code all right and I outlined uh and you can reference this to in the GitHub repo over here but here's like a bunch of different environment variable options and the explanations

For what they are this is how I set up my environment variables very very very simple this is literally like the Bare Bones really all you need is the auto uh gen model which for me I just put GPT 40 and then the autogen model API key now if you're using open AI you don't need to put a different base URL so you can remove that uh variable if you're using a different uh model or a different um provider llm then

You're going to want to have the autogen model base URL here and then put the base URL in the environment variables and then yeah so I just have the API key here now if you're not using playright and you're using like uh Chrome and you want to use your Chrome directory then you would include this environment variable browser storage dur and you would just put the directory to your um browser right there so so you could get that by going into

Your Chrome and then just searching Chrome version in your Chrome uh search bar and that would find the path to your profile and then you could just use that path in the uh Direct right there in the environment variable okay once you have all that set up you have your environment variable set up then you can run the code and you can do so just by running this Command right here now for Mac users you can run this command all right

So I'm going to go ahead and rerun this right now so python dm. Main and if everything goes well it should open up the Chrome browser so you'll see here it's creating the auto wrapper user planner agent browser nav agent browser nav executor okay okay and it opened up our browser right here so as you can see here it's just a normal browser but we have a little uh AI robot icon right here so if I go ahead and click on

This you can see we got our a agent e now this somewhat looks similar to um kind of agent Q if you saw agent Q how they have the Chrome extension this kind of looks something similar to this but uh let's go ahead and just ask it something so let's say all right so my PR is please search for Josh pook on YouTube find the most recent viewed video in the last 7 Days on the most recent view video in the

Last 7 Days uh and let me know how many comments likes and and Views it has and click on it and pause it so you can see here it's already gone to YouTube are you tired of pouring thousands of dollars into appointment Setters only to watch leads slip away imagine having a team of elite sales agents booking qualified appointments for you around the clock no more wasted time on training no more frustration with performance and no more drain your budget on

Inconsistent and expensive call centers introducing stride agents AI powered appointment centers that work 24/7 never get tired and book appointments while you sleep trained on thousands of successful conversations our AI agents El perform human teams at just oneth of the cost join the ranks of businesses that doubled their appointments and booking rates in just a matter of weeks don't get left behind in the AI Revolution visit stride agents.com now and transform your entire sales process with cutting Ed Edge AI technology

It's time to accelerate your stride with AI agents it's already searching Josh pook and you can see here as it's doing so there's like a bunch of um code writing right here in our terminal we can see it's executing right here can see different autogen tools or running so pretty cool stuff and we can literally see here it generated a plan right so step by step uh eight different steps right here you know everything we've asked for next step next step

Is search for Josh next step is filter to show results for the last s days so it did so and I am quite honestly impressed by one the speed I mean it's not lightning fast but it's pretty pretty fast and also two the accuracy so as you'll see here it's filtering by this week okay now it's identifying the video with the most views from the filtered search results okay so now it's extracting the number of comments likes and videos titled for

The bolt. new video which yes is my uh most viewed video in the last week okay now it's clicking on the video so now the video is playing let's see if I can pause it okay so it did pause it so it got every single step correct like it did not fail which is pretty impressive to be honest um so yeah now it is done let's just see the response okay so it gave us a response so this video right here

The exact title name has 19k views 562 likes and the number of comments is not explicitly available uh that's the one thing let's see so yeah I guess it had to scroll down to find the comments so maybe if I um specified that um it would have got it but I think that's pretty impressive um okay so now I think this one's a bit more challenging I said search for Toronto real estate and provide me a list of 10 homes in

Toronto that are near the CN Tower and provide me with their prices go ahead and test this okay so it developed plan just like we talked about so now it's gone to realtor.ca and it's going to search for homes in Toronto and then use the filter to narrow down the searches to homes near the seen Tower and then extract the top 10 and tell us the pricing okay so wow okay so we're in Toronto now we got our map and now

It's going to check and see if there's any options to filter to near the CN tower which probably won't have any so click on filters button to EXP the specific filter options for narrowing down searches into the CN tower which it probably doesn't have oh maybe it does in keywords I don't know okay so it looks like it pretty much found the CN tower now it's extracting the top 10 homes here and getting there okay so it already got it like

That was really quick um we got you know 405 300 Front Street and like this is exactly right near uh the CN tower we got 650k 6 00k whatever so 950k um yeah that was that probably all in all took like 30 40 seconds under a minute we got all that um and I thought that one was going to be a little bit more difficult but you can see here you could use this for somewhat scraping you could use this for

A bunch of different stuff and this is really just a start I think if you you know use this and maybe customize it a bit maybe integrate it in an app or a project I don't know I haven't oh so there actually is um some information here on their GitHub in terms of advanced usage you know launch via web endpoint um different information here such as customizing llm parameters uh talking about open source models as well and um a bunch of

Different stuff some videos here so I would definitely recommend checking out their GitHub repo um for more details also too uh one other thing they do have a Discord as well so I would recommend checking out Discord channel right here I'll leave a link to this as well in the uh document so you guys can access that at the top um so if you are planning on building something more custom um you could probably reach out for there for support if

You need anything or see what other people are building in there so other than that I will be playing with this tool a lot more as well so let me know what your thoughts are around about this in the comments Down Below guys if you've used this tool before um and you maybe used agent Q let me know what your thoughts are between what agent is better and let me know what your favorite AI agent is so far I will be

Covering more if there's one I missed let me know in the comments Down Below guys other than that guys that's pretty much it for this video I just wanted to cover agent e for you guys because I think it's a really cool tool if you're new to the channel we make videos every single day on AI automation business growth Marketing sales Etc you got some value here and you like this type of content make sure to smash that like button and

Subscribe to stay up to date with the daily uploads also too guys thank you so much for the recent 8K subscribers we just hit 7K 7 days ago and now we're already at 88k so I appreciate recent support I'm really grateful for all the uh new subscribers and for all you guys watching uh even if you're not subscribed I'm very grateful for you guys as well definitely motivates me to keep uh increasing the quality of content and providing you guys with

More value there's a couple videos I have been working on actually pretty late right now cuz I've been working on this one video so I'm excited to share with you guys hopefully I will have that done by tomorrow it's been a few days now but if you're new to the channel you haven't already join our free community stri community.com our free Facebook group and free Discord Channel link to that will be in the description down below also too guys if you

Run a business or you run a call center you're sitting on a bunch of leads and you don't want to call them or you have you know lazy sales reps or you have appointment CS but you're just spending too much money or you're not getting quality results and check out stride agents.com it's our AI appointment cters via voice via text via email via socials check that out in the description down below you can heer some examples and it just really works

AI appointment C is 24/7 on autopilot speed to lead is key so if you're not calling your leads right away the chances of closing of them are drastically decreasing so check out stride agents and if it looks like it could be a fit then just book a call and we can see if it would make sense to potentially work together other than that guys I will see you in tomorrow's video keep hustling keep grinding and of course guys accelerate your stride

Take care