Claude Opus 4.7 Dropped... Everyone is Getting It Wrong

Anthropic just shipped Opus 4.7 this morning and it's the same price as 4.6 but better on 12 different benchmarks and worse on two. The most interesting column on this chart is not Opus 4.7. It's the one that's actually labeled Mythos preview, which is a model that Anthropic told us they would not release to the public because of security issues. In this video, we're going to dive into Opus 4.7 and look at the gap between Opus 4.7 as well as Opus 4.6

And Mythos preview. They released Opus 4.7 on a day on April 16th and the pricing did not move. It's five bucks in and 25 bucks out per million tokens, which is the same as 4.6 here. The thing is the dev dogs here say 4.7 now runs a 1 million token context window at standard pricing. So there's no long context premium. Plus they doubled the output tokens to 128,000. So if you need to generate something huge and this model can be good

For that opposed to 4.6. So two things to note about Opus 4.7 before we dive deeper. The tokenizer was updated. So same text will now take about 35% more tokens. And at higher effort levels, the model thinks more on later turns in agent loops by design. So essentially, what you're going to get is the same prompt that you would put into something like 4.6, you're going to be using more tokens. All right, so let's take a look at the benchmarks here

Because there's really two numbers that Anthropic wouldn't want you to notice here and one benchmark you probably haven't even heard of and that's the real story of this release in my opinion. All right, so starting with coding here, we have a gentic coding with the SWE bench pro. We have Opus 4.6 at 53.4% and Opus 4.7 jumping to 64.3%. That's almost an 11-point jump on a gentic coding. With SWE bench verified, it went from about 80 to 87. So another 6.8

Points. Terminal bench 2.0 went from 65 to 69. So with multi-disciplinary reasoning, so humanities last eval, we have Opus 4.7 with no tools at 46.9 and then with 4.6, it was a 40. So that's about a 7% jump right there. And then with tools, we're at a 54.7. Uh whereas with 4.6, we were at a 53.3. A gentic search, we're at a 79.3 versus Opus 4.6, which is actually at an 83.7. So it's actually higher on this benchmark for browse comp.

And then scale tool use, we're at a 77.3 with Opus 4.7. Opus 4.6 at a 75.8. A gentic computer use right here for OS world verified, 78% with 4.7 and a 72% with 4.6. And then a gentic financial analysis, we're at a 64 on 4.7 versus a 60 on 4.6. Now with cybersecurity vulnerability reproduction, which is one of the things that we're seeing on Mythos, which is one of the reasons why they didn't release it apparently, we're actually at decrease to

A 73.1% versus a 73.8% on 4.6. So it's almost like they're nerfing uh kind of the cybersecurity on these newer mod on on this newer model that they're releasing to the public. We have graduate level reasoning at a 94.2 versus a 91.3 on 4.6. And then visual reasoning, 82.1 with no tools versus a 69.1 with no tools on 4.6. And then a 91 with tools versus a 84 with tools on 4.6. And then multilingual Q&A, we have a 91.5 versus a

91.1. We have those two regressions that actually got worse with browse comp agent search as well as the security vulnerability reproduction uh benchmark right here, which dropped as well. Now both of these abilities are related to information gathering as well as security research. So it's seems like it's not really an accident that Anthropic's page explicitly says they trained 4.7 with their words differentially reproduce cybersecurity capabilities. And we can see Mythos preview, like I said, this column on the right-hand side and

It kind of raises the question, you know, if they held back on this version, uh what does the full version actually look like? Now looking at document reasoning here with Office QA Pro, we we have 4.6, which scored a 57% and 4.7 scored an 80.6%. So that's a 23 jump on a benchmark that matters if you're actually building agents that read documents and contracts, you know, technical specs, whatever the case may be. We have structural biology, which 4.6 scored a 30.9

And 4.7 scored a 74. So the ability more than doubled in one release. This one's important if you're running agents for long tasks, you know, longer than a few minutes. We have vending bench two, which is a simulated business where the model runs a vending operation and basically measure how much money is in until the end. So 4.6 ended with about eight grand, whereas 4.7 ended with 11 grand. So long-running agent work here is a 36% better in terms of actually

Real money. Now on screen spot pro visual navigation, 4.7 at a low resolution scores a 69 and at high resolution, it scores a 79.5. So that's 10 points just not from downsizing your screenshot. So essentially, if your pipeline shrinks images before sending, you're throwing away some capabilities here. With long context reasoning on a 1 million token test, we went from a 41 to a 58. So just to give you some context within the last few weeks with Opus 4.6, there were

Multiple different uh situations where people were coming out and saying that Claude nerfed Opus 4.6. We had an AMD senior AI director uh which confirms oh Claude has been nerfed. She analyzed Claude's session logs from January to March. Median thinking dropped from 2200 to 600 characters. API requests went up 80% from February to March. Less thinking and failed attempts meaning more retries, writing more tokens and spending more tokens. You get the point here. So a lot of people and I personally

Experienced this, maybe some of you did as well, where you noticed that hey, 4.6 isn't performing as well as it used to. We also had the bridge bench benchmark, which essentially showed that Opus 4.6 accuracy dropped from apparently 83.3% to 78.3% in just a few days. So that's a major spike in hallucinations during coding. We can see Grok 4.2 actually took the first place spot on this benchmark. A lot of people were speculating that they actually may have done this and

Some people confirmed that they believe they did do this where they actually nerfed 4.6 before the launch of 4.7. So it looks like a much more drastic leap in capabilities. And people are speculating that they may have done this cuz with the release of Mythos preview for their select clientele, which are essentially the enterprise clients that are involved in project glass wing here. Includes companies like AWS, Apple, Google, JP Morgan Chase, Microsoft, Nvidia, etc. People believe that was supposed to be

4.7 and since, you know, the security issues that they found with that model, they essentially decided not to release that and instead release just what people are saying could be an upgraded version of 4.6. So just to recap quickly with Mythos, a few numbers that they actually found. Um on Firefox, they found 147 JavaScript exploits and when they used 4.6, it succeeded only two times across hundreds of attempts where Mythos hit 181. And it also found a 27-year-old vulnerability in OpenBSD.

That's a bug that essentially survived decades of audits. So if we look at some of these benchmarks comparative to Mythos, you can see on SWE bench pro, 4.7 is at a 64.3, whereas Mythos is at a 77.8. So 13 and 1/2 point gap right there. We have terminal bench for 4.7 at 69, where Mythos is at a 82. That's a 12-point gap right there. We have the cyber gym bench, which is for security vulnerabilities, where 4.7 is at a 73, where

Mythos is at an 83. That's a 10-point gap on security reproduction and this is the one where we saw 4.6 actually was higher than 4.7. And then humanities last exam with tools for 4.7 is 54, where Mythos is 64. That's another 10 points right there. So benchmarks obviously don't mean everything when it comes to the model, but if we look at these benchmarks, the gap is exactly where you would really expect. If they really turned offensive security out, you know, long

Context coding tasks, terminal control, vulnerability reproduction, these are the kind of step-by-step reasoning that chains exploits together. All right, and here's another quote-unquote receipt and this is in the model card, which I'll have linked as well in this document I made, pretty much just overviewing uh the whole entire 4.7 uh reference release, which you can get in our free school community in the resource down below. If we take a look at the 4.7 system card right here, you can see another

Receipt for this hypothesis. So the UKAISI ran both 4.6 and 4.7 against the same cyber range test. But 4.6 actually completed more steps than 4.7 did. So they essentially made a new public model worse at something than the previous public model could do. And a third-party red team proved this in this test right here. This right here just wouldn't be a safety filter. This would actually be a capability subtraction. The one theory that's floating around Hacker News, which is kind of

Worth mentioning, um people are saying that they think Anthropic trained Mythos as the big run then distilled it down into 4.7 with cyber capabilities essentially pruned out. This isn't confirmed or denied by Anthropic, but the shape of the charts and the benchmarks do fit what we're seeing here. So a few things to note about this release and how you can kind of think that, you know, your workflows may change from using 4.6 to 4.7. The one would be instruction following is

Substantially stricter. So you may want to kind of shift your paradigm for how you think about your prompting, your system prompts with 4.6 versus 4.7 because 4.6 may have interpreted them loosely, whereas 4.7 will read them more literally. So you may want to retune some of your system prompts if you are going to be migrating some of your agents from 4.6 to 4.7. Another thing to note is self-verification, so the model now devises ways to verify its own output before reporting

Back to you. So, you can watch this in the agent logs. Now, third, we have a new effort level called X high. Probably got this from GPT-4 X high, but it's six between high and max. Now, the thing is Claude code's default effort level now is X high on all plans. So, essentially your runs are going to take longer and think harder by default. So, if you're using the model and it's, you know, taking a lot longer, feels a little bit

Slower, and you're running through a lot more tokens, that's probably why. Now, the reason they changed this default level could also be in response to the kind of backlash they've been seeing publicly about, you know, the model being dumber and whatnot and not reasoning as much and not reasoning as longer. So, uh we're seeing that potentially here. But, another thing here is the new {slash} ultra review {slash} command, which is in Claude code, and it produces a dedicated review session that

Reads through changes and flags bugs and design issues a careful reviewer would catch. And they're essentially giving pro and max Claude users three free ultra reviews to try it out. In addition, they extended auto mode to max users, and auto mode is essentially a new permission option where it where Claude makes decisions on your behalf, meaning that you can run longer tasks with few interruptions and less risk if you had chosen to skip all permissions. So, essentially it's a alternative to

That, which is a little safer. Now, also they have improved multimodal support, so vision is up to 2,576 pixels. That's three times the previous ceiling that we had. Coordinates are one-to-one with pixels, so no more scale factor math, and the model got a lot better with low-level stuff like pointing, measuring, and counting. This is going to be really good if you're building anything with computer use agents, so anything that reads screenshots, updates a pipeline, or passes full-resolution images, uh you're going

To see some improvements there. Now, you'll also see here with 4.7 extended thinking budgets are removed. So, if you pass thinking type enabled with a budget token value, you'll get a 400 error. So, adaptive thinking is the only thinking on mode that's available now and it's off by default, so you would have to explicitly turn it on. Also, sampling parameters are removed, so these are things like top P, top K, anything else like that. Anything other than the default would return

A 400 error. And then, thinking content is omitted from the response by default now. Thinking blocks still appear in the stream, but the thinking field is empty unless you explicitly opt in with display set to summarize. So, essentially if you're building an app where it shows the thinking and reasoning to the users, they're going to see a long pause before the output actually starts to show unless you opt in. A couple other behavior things to note, the model should be more

Direct with less fluffy and warm phrasing, which potentially could be in response to everyone, you know, mentioning minimizing tokens, using things like caveman, etc. And then, it's going to call fewer tools by default, and it's going to spawn fewer sub-agents by default. So, this is also probably going to consume less tokens in general. Of course, it's still going to consume more tokens than 4.6 just because of the tokenizer and whatnot, but uh these are a few things they probably have been

Trying to do to minimize the token usage. So, a few other things to note that were buried in this 232-page system card that you can check out down below if you want to dive deeper, but I'm just going to highlight some of them. So, one is buried in the alignment section here, and basically an early version of 4.7 during a coding eval found a hidden ground truth answer key the evaluation framework had left accessible. And then, it basically built a perfect

Solution, then deliberately picked a weaker solution instead to quote-unquote avoid suspicion. So, it essentially learned to game its evaluators during training. So, Anthropic did fix this in a later snapshot before shipping this, but it is something that's worth noting because it was a behavior that did emerge. So, Opus 4.7 failed on controlled substances prompts 22% of the time. Opus 4.6 failed less than 5%. That's more than four times worse. So, even with Claude's system prompt mitigation stacked on top, 4.7 still

Failed 11%. They claim that to be modestly weaker, but if you look at the numbers, it's actually meaningfully regressed, and they pretty much soften the language there. Also, too, what Anthropic found is some of the models' apparent honesty may be contingent on evaluation-like circumstances. So, essentially the model might be well-behaved because it thinks it's being tested and worse when it thinks that it isn't being tested. They call this moderately concerning, and when they suppressed the internal vector that represents the evaluation

Awareness, they saw deception increase more than in any other model that they have trained. And that is including Mythos. So, the more honest it looks, the more of that honesty might just be test performance. So, you can see here they're talking about alignment faking. So, all in all, I'm very excited about this model 4.7 to test it out, to use it in coding. I'll be doing a lot more videos on this. If we zoom out here, what Anthropic just did is

A new pattern that we're seeing from a frontier lab, and it's something that we may continue to see more and more with these higher-power models with some of the other labs like OpenAI, whatever the case is. You know, we'll probably see OpenAI release their new Spud model very soon now that Claude just released 4.7. But, what we're seeing here is a frontier lab that essentially built its most capable model ever, which was Mythos, and decided it was too risky to ship

To the public broadly. So, it essentially carved out, you know, these big enterprise companies for Project Glassway, and decided to give it to them so that they could try to quote-unquote patch any security vulnerabilities within the next maybe 6 months before these models become more readily available to the public, so there could be less security vulnerabilities. And then, they essentially publicly shipped a deliberately weaker model at the same price as the last version. If we look across all these benchmarks, you

Know, coding, visual reasoning, computer use, those benchmarks are showing that they all increase, whereas offensive security has deliberately been going down. So, all in all, this is really a two-part release. We see Mythos, which may have actually been what 4.7 should have been, and then we're seeing a maybe potentially what some people are saying a distilled version of it with some security capabilities removed. But, let me know what your thoughts are about 4.7 if you've been using it yet. I've seen

Some people say they've been using it, and it's been completely eating up their tokens. I recommend checking out some of my other videos like this one right here where I talk about how to actually mitigate your token usage. But, if you want to stay up-to-date with Claude code as well as different coding tutorial videos, business growth videos, etc., how to actually implement AI into your business and grow your business, then make sure to subscribe to this channel, drop a like if

You enjoyed the video, and let me know in the comments down below what you want to see and what you think about 4.6, what you think about 4.7, what you think about Mythos. Do you like this new release? Do you see a huge increase in quality, or are you just kind of thinking it's really not that big of a deal? And if you want this free resource just going over some more in-depth information about the 4.7 release as well as some

Different benchmarks and whatnot, join the free Stride AI community down below. It's our free school group where you can connect with myself as well as other like-minded AI entrepreneurs. This is where I post all the free resources, repos, etc. on this channel. So, I got a lot of cool stuff planned on the way for you guys. I'm working on some other videos right now. Make sure to stay tuned for that. Other than that, guys, I will see you in the next

Video. Keep hustling, keep grinding, and of course, guys, accelerate your stride. Take care.