From the Announcement of Claude Fable 5 to Case Studies of Japanese Companies: Code with Claude 2026 Tokyo Report | Thorough Explanation for Beginners [Part 2: Development Practice]
How to reduce the cost of AI agent development by 90%
A thorough guide to capability curves, evaluation, and context engineering
*This article is the final installment of the Code with Claude 2026 Tokyo Complete Report (3 parts total), which is the [Part 2]. The
[Part 1] covered the new model "Claude Fable 5" and its new features, while the [Part 1.5] covered vivid implementation case studies from top Japanese and American companies such as Rakuten, Mercari, Mizuho, NRI, and Canva. Chapter numbers are continuous throughout the entire series.
[Part 1.5] looked at companies achieving phenomenal productivity improvements by incorporating AI into their organizations not as "just a chat tool" but as "autonomously working agents" (colleagues). So, how do developers on the front lines control AI behind the scenes, which tends to run wild because it is too smart and sometimes consumes enormous costs? This article will help you understand the following:
How dramatically AI model capabilities have grown over the past year, which you can confirm through the voices of the Anthropic development team and the latest research data from third-party organizations.
Why "Evaluation" (Evals) is called essential (unit testing) instead of traditional testing in the AI era, and the thinking behind it.
The magic technology that reduces AI API usage costs (communication fees) by up to 90%, the mechanism of "Prompt Caching", which you will be able to explain to others.
To prevent AI memory from overflowing, you will be able to judge the three tools of "Context Engineering" (context saving techniques) and how to use the "Advisor Strategy" to build a cheap and smart AI team.
How non-engineer business professionals should face this "era where AI builds software by itself," and action guidelines for tomorrow.
The roadmap for this discussion is as follows. First, we will confirm what kind of terrifying curve AI capabilities are currently growing along, and think about how developers should prepare for the "next future model" (
Chapter 6). Next, we will learn the gritty yet powerful operational know-how (cost engineering) to draw out the power of that strong AI "cheaply, reliably, and without overflowing" (
Chapter 7). Finally, we will conclude the entire series by thinking about what this change means for "non-developers," based on the surprising anecdotes from community sessions where non-engineers are winning hackathons one after another (Final Chapter
).
*Note that while advanced technical terms appear in this article, they are all explained with intuitive analogies. Even if you are a non-engineer, please try reading while focusing on the essence of "organizational barriers in AI adoption" rather than the technical mechanisms themselves.
Chapter 6 | Riding the Capability Curve
Development philosophy based on the premise of the "next model"
■ From 62% to 88% in one year, and then "saturation"
The lecture by Research Product Manager Theo Chu began with the introduction of a certain "number" that tells the story of the tremendous evolution of AI.
*Some technical terms will follow from here, but rest assured that the essential meaning will be explained with easy-to-understand analogies later.
Over 80% of code internally at Anthropic is merged by Claude.
(Free translation) In Anthropic's internal development, over 80% of code is merged (meaning it passes strict reviews and is integrated into the production system) by Claude.
The fact that the company developing the AI itself is already entrusting 80% of its software development to AI is surprising enough, but Mr. Chu further presented a graph. The subject was "SWE-bench Verified". This is the industry's most trusted benchmark (performance test) that measures whether an AI can fix actual bug reports on GitHub and pass the tests. The score for the model from a year ago, "Sonnet 3.7," was around 62%. That reached 88% with "Opus 4.8," and
with the latest Mythos and Fable models announced today, this benchmark has finally become "saturated"—meaning it has essentially solved the prepared test questions with a near-perfect score. Mr. Chu added a very important perspective regarding this graph.
From 62% to 88% in just 12 months means that Sonnet 3.7 actually failed three times as often on these tasks.
(Free translation) The fact that it grew from 62% to 88% in 12 months, conversely, shows that Sonnet 3.7 failed about three times as often on these tasks.
He pointed out that it should be viewed not as a "linear growth" where the score increased by 26 points, but as a "curved growth" (exponential) where the failure rate plummeted to about one-third, from 38% to 12%. This view that "capabilities will continue to grow rapidly" is not just Anthropic's self-promotion (marketing). The paper "
Measuring AI Ability to Complete Long Tasks" published in 2025 by the third-party AI safety research organization METR scientifically supports this.
METR proposed an indicator called "the length of tasks an AI can handle with a 50% success rate" (how many minutes of work it would take a human to do) and named this the "Time horizon". According to the paper's analysis, this "time during which AI can work autonomously" has doubled approximately every seven months from 2019 to 2025, and if this tremendous trend continues, it is predicted that "within five years, AI will be able to automate much of the software work that would take a human one month."
The background behind Diane Penn saying in the keynote of [Part 1] that "we look at the time horizon as an indicator to measure the value of a model" and Mr. Fujii of Mizuho saying in [Part 2] that "we are working backward from the superintelligence of five years from now" was precisely this solid research lineage (capability curve) backing. However, it should be noted that this is merely a prediction (extrapolation) based on past trends and is not a complete guarantee of the future.
■ Three Evolutions of Models and Three Tactics for Developers
So, within this capability curve, how exactly is AI becoming smarter? Mr. Chu organized the evolution of AI into the following three points.
① Planning before acting
Old models would be like me with IKEA furniture. They would jump right in, not look at the instruction manual... Now models actually plan first. (Free translation) Old models were like people who start assembling IKEA furniture without an instruction manual; they would just start moving immediately. Now, they plan first before proceeding.
Previous AI would start writing code immediately and realize mistakes later, but now they think about the specifications (blueprints) first before taking action, which results in less rewriting and faster, cheaper completion.
② Error recovery
In the past, AI tended to fall into a "doom loop" where it would return to the same solution even after being pointed out its mistakes. However, current models can receive feedback from the environment, try different paths, and recover on their own.
③ Running over longer horizons
Even in long-term tasks on the scale of 1 million tokens, they no longer "lose the plot" of the original instructions midway through.
As AI gains autonomy, how should we developers act? Mr. Chu presented "three tactics (prescriptions) for developing for future models, not the past."
[Tactic 1] Establish Evals as "Unit Tests for the AI Era"
Include not only tests for problems that can be solved now, but also intentionally include "difficult problems that today's models cannot yet solve" in the tests (Evals). Then, on the day a new model is released, run them, and the moment you observe that "that task which couldn't be solved before can now be solved!" is the perfect signal to release your company's new product.
Conversely, if you keep tests that only collect problems that current AI can easily solve (= a saturated state), you will not notice the true power of new models. Mr. Chu warns of this.
Hey, on my eval it only really improved by 1%. I don't think this model is that much better. [...] your eval might be saturated.
(Loose translation) "Wait, my evaluation tests only showed a 1% score increase with the new model. Isn't it not that much smarter?" ...If you feel that way, you should suspect that it is not the model, but the test you prepared that is "saturated" (saturation).
[Tactic 2] Shrinking the Scaffolding
"Scaffolding" (scaffolding) refers to the prompts and control code that humans build around an AI to make it behave as intended. Mr. Chu shared a failure story from his company. One day, after switching to a new model, a report came in that "
Claude stopped following the specified citation format" (it broke). However, upon investigation, it turned out that "instructions for an old citation rule that no one used anymore were left deep inside the system prompt". In other words, the truth was that "
because the model became smarter and its 'faithfulness to instructions' improved, it dutifully followed the old scaffolding (instructions) left by humans, causing a malfunction". When you pile up "failure avoidance rules" from an era when AI was still immature, they actually become a hindrance for a smarter new model. The lesson is that prompts should be written simply as "intended goals" rather than a "list of things not to do."Prompts should be written simply as "intended goals" rather than a "list of things not to do."
is the lesson.
[Tactic 3] Giving the Model 'Room to Think' and 'Means of Action'
Models can now recover from errors on their own, but to do so, they need a way to know that they made an error in the first place. Specifically, it involves utilizing "
adaptive thinking," which allows the AI to think as much as necessary when needed, and "effort dials" that allow you to adjust how deeply the AI thinks like a knob.
And, for example, in app development, it means giving the AI "means to verify its own deliverables itself," such as a 'computer operation tool' that can actually click on the screen to check performance. Closing the AI's 'agent loop' (creating a self-contained cycle of creating, checking, and correcting itself) is the biggest key to riding the capability curve.
[Glossary] ○ Benchmark Saturation
A state where test questions become too easy, everyone gets a perfect score, and it becomes impossible to measure further differences in ability. Just as you cannot compare people who can read the very bottom line of an eye chart, you need to "switch to a more difficult test" (update Evals) once saturation occurs. ○
Doom loop: A "death loop" phenomenon where, when an AI stumbles on an error, it repeats the same incorrect procedure infinitely no matter how many times it tries again, consuming only wasted communication costs (tokens). ○
Scaffolding
The entire system of prompts and tool controls written by humans as support to ensure the AI does not get lost. The smarter the AI becomes, the more these training wheels need to be removed (shrunk).
Chapter 7 | Engineering Costs:
Three Techniques to Reduce API Fees by 'Up to 90%'
In the era of autonomous AI agents, "cost and memory management" is just as important as the model's intelligence. When you entrust long tasks to AI, API usage fees (communication costs) skyrocket due to the exchange of vast amounts of documents and tools, and the AI's internal memory (context) quickly hits its limit. In this Chapter 7, we will introduce "
three techniques to dramatically lower API fees and prevent AI memory from hitting its limit" provided by Anthropic.
■ Prompt Caching: 'If you take one thing home from this talk'
Brad Abrams, Lead for Platform Product Management, asserted strongly to the attendees at the beginning of his talk.
This is the single most important takeaway of the talk: how you do prompt caching.
(Loose translation) The most important point in this talk is how you perform "prompt caching".
The mechanism is as follows. In long dialogues with agents, a 'long preamble'—such as a 'system prompt' that defines rules or 'previous conversation history'—is repeatedly sent with every API request. Recalculating this from scratch every time is expensive and slow.
Therefore, the calculation results for the preamble are temporarily saved (stored) on Anthropic's server side, and for the next request, only the 'difference from the previous time (newly added instructions)' is processed. This is Prompt caching (reusing prompt calculation results).
The power of this is immense. According to the official document 'Prompt caching', input data that hits the cache (where memory is reused) becomes '90% off the price' (in other words, one-tenth the price), and response speed is also up to 85% faster. Moreover, cached data does not count toward API rate limits (usage caps within a certain period).
According to Mr. Abrams, global AI services like Perplexity, Cursor, and Replit are thoroughly implementing ways to keep this cache hit rate extremely high.
If these customers didn't have such a high cache hit rate, we couldn't even serve their workload because there's just not enough compute without prompt caching.
(Free translation: If these customers did not have such a high cache hit rate, without prompt caching, there would not be enough compute resources to even support their workloads.)
As a guideline, if you are developing an agent that runs for a long time, you should aim for a cache hit rate in the 80% range.
A 'dashboard' that shows your app's hit rate has been added to the developer console, and an official skill has also been introduced where, if you ask in the Claude Code chat screen, 'improve my cache hit rate' (improve my cache hit rate), the AI itself will fix the configuration file. In the actual demo, a fictional app that had a 0% hit rate had cache settings implemented on the spot, and it was brilliantly shown improving to 58% immediately.
*Please refer to the 'Getting started' section.
[Terminology Explanation]
○ Cache
A general term for a mechanism that keeps calculation results or retrieved data on hand so that they can be reused next time without recalculating. The reason the price is 90% off is because 'Anthropic can also skip wasteful calculations, so the electricity savings can be passed on to the user'; it is not a limited-time discount campaign, but a discount system based on physical rationality.
■ Context Engineering: A technique for saving the limited resource of context
Next, Mr. Abrams spoke about 'Context Engineering', the latest design technique for saving AI memory (context window).
Context engineering is the discipline of deciding what belongs in Claude's context.
(Free translation: Context engineering is a method/field for deciding 'what to include in Claude's context'.)
An AI's 'context' (memory capacity) is often compared to the 'size of a desk you work at.' Just as you cannot do new work if you spread too many unnecessary documents on your desk, the AI's brain will quickly become overwhelmed.
The task prepared for the demo was to have the AI create a management dashboard for a fictional hero dispatch company, 'HeroCorp.' Initially, even using its massive 1 million token memory limit, the AI could only process one of the four management goals before crashing.
However, when all three of the following 'tools' were introduced, it easily processed all goals within the 1 million token limit, and still had memory to spare.
[Three Tools ①]
Tool Search Tool
Put away tools you don't use on a shelf
While it is healthy to give a capable agent 100+ tools, it is wasteful to load all 100 'thick instruction manuals' into the AI's memory from the start. After all, you only use about 10% of them in actual work.
Therefore, we use a method where only the 'tool searcher' is placed in the initial memory, and the tool manual is searched for and retrieved only when the AI needs it. According to the lecture, the development platform company Lovable reduced token usage by 10% with just this change, and also gained the benefit that 'the model's judgment itself improved' because the memory became lighter.
[Three Tools ②]
Programmatic Tool Calling
Copy only the necessary pages and put them on your desk
Suppose the AI uses an 'email search tool' and the full text of 100 emails is returned. However, it is often the case that you only really need the 'subject line of one email.'
Therefore, the vast raw data is first received outside the AI's memory (in a sandbox environment), and using a small Python program written by the model itself, 'only the necessary few percent of data' is extracted, and only that result is put into the AI's memory.
According to Anthropic's official engineering blog 'Introducing advanced tool use on the Claude Developer Platform', the Q&A site Quora reported that by switching to this method, they reduced average token usage in complex research tasks from 43,588 to 27,297, a full 37% reduction.
[Three Tools ③]
Compaction (compression by summarization)
Summarize finished documents into a single short memo
Even if you make full use of 1 and 2, an agent working for a long time will eventually use up its memory limit.
Therefore, you set a 'threshold' (e.g., 'when 400,000 of the 1 million tokens are used'). When that line is reached, the AI's work is paused, and a 'short summary' is automatically created, stripping away past progress that is no longer needed as judgment material.
Then, work is resumed with that as the new memory. The data analysis company Hex had previously built a similar complex mechanism in-house, but by switching to this official Compaction feature, they were able to delete 300 lines of management code.
[Terminology Explanation]
○ Context Window
The upper limit of the amount of information (memory) that an AI can "keep in view" and process at one time. It is very easy to understand if you compare it to the "size of a desk you work on."
○ Tool Search Tool
"Do not spread tools you are not using on the desk; keep them in a shelf."
○ Programmatic Tool Invocation
"Do not put thick documents on the desk; copy only the necessary pages and put them on the desk."
○ Compaction
"Clean up finished documents from the desk and summarize them into a single short memo". Techniques like this for constantly organizing the top of your desk (context) are what context engineering is all about.
■ Advisor Strategy: Assigning a Senior to a Junior
The final cost-saving technique is how to combine models with different capabilities. Mr. Abrams explained this strategy by comparing it to common sense in real-world development teams.
You can make a junior engineer on your team more productive just by giving them access to a senior engineer... The same thing is true with models.
(Paraphrase) Just by giving a junior engineer on your team an environment or opportunity to ask a senior engineer a quick question, their productivity improves dramatically. The same is true for models (AI).
Entrust routine tasks to inexpensive, high-speed models (Haiku or Sonnet), and provide them with a tool to consult (get advice from) high-performance models (Opus or Fable) only when they are truly stuck or need to make important decisions.
In the demo, a case was shown where an inexpensive Sonnet model judged that "this business negotiation is going well," but the Opus model, upon being consulted, corrected it by saying, "No, there is actually a danger signal," based on an "oversight at the end of the meeting minutes," successfully saving the deal (closing the negotiation).
The new model "Fable 5" is priced high in exchange for its overwhelming intelligence ($10 input / $50 output per 1 million tokens).
Therefore, this advisor strategy of "using expensive models only for key points where deep reasoning is truly necessary" is an extremely practical method for optimizing costs without sacrificing quality. You should also notice that it is [Part 2] the exact same philosophy as Canva's "sub-agent approach" introduced in the previous section.
At the end of the lecture, a list of features announced by Anthropic just in the first half of 2026 was displayed on a single slide.
WIF (Workload Identity Federation)
Eliminates the need for API keys themselves, an authentication method that reduces the risk of key leakage in source code to zeroFallback API (Fallback feature)
Fable 5 A feature that automatically switches to another model to prevent processing from stopping if a safety mechanism (classifier) is triggered in a powerful model like this
The evolution of the platform is tremendous, and it was filled with such excitement that the changes of just half a year could not fit on a single slide.
Final Chapter | The Era Where "Business Experts" Create Software
Proposals to Readers
Closing the day of the event was a session by Anthropic's Japan Developer Community Lead, Junichiro Tsuji. Having been at the forefront of software development for over 20 years—at Sony for PlayStation, at Google for search and game infrastructure (Stadia), and at Roblox for developer support—Mr. Tsuji summarized the changes since last autumn by stating that
rather, "the opportunity has come for non-engineers".
I think the barriers to creating software have almost disappeared. [Omitted] I think we have reached an era where, if you can verbalize 'I want something like this,' you can write almost any software.
Just as spreadsheet software made "complex calculations," the internet made "information dissemination," and smartphones made "high-performance computers" accessible to everyone, the idea is that now "creating software" itself has been opened up to everyone. The constraints of development have completely shifted from "can you write programming code?" to "can you put the problem you want to solve into words?".
■ Hackathon Winner is a "Lawyer Who Has Not Written a Single Line of Code"
This is supported by the surprising results of the hackathon (a short-term intensive development event) for which Anthropic recruited participants from all over the world, Built with Opus 4.6: a Claude Code Hackathon (February 2026). Out of approximately 13,000 applicants, 500 were selected to work on their own challenges for a week. As a result, among the five winning teams, four were not engineers.
The winner was a lawyer from California (Mr. Mike Brown). He focused on the social issue that "in that area, 90% of residential building permit applications are rejected, and it takes longer to get a permit than to build the house."
He devised a solution that suggests 'how to write documents that pass inspection' in just 20 minutes simply by dragging and dropping drawings and correction instructions. He managed to build this sophisticated system, which runs numerous sub-agents in parallel in the background, in just one week.
I can't believe I won. I didn't write a single line of code, and I didn't read a single line either.
The other winners were similar. The doctor who took third place created a service called "Post Visit" that delivers easy-to-understand explanations to patients after a consultation about "what the doctor based their diagnosis on and how they reached it." The winner who performs road assessments for the Ugandan Ministry of Transport reduced investment decisions that previously took five weeks to just five hours by having AI analyze dashcam footage.
■ Fusion of "Top-Down" and "Bottom-Up"
The same change is happening within companies. At Zapier, a fully remote SaaS company with about 800 employees, Claude usage that started with voluntary employees developed into a company-wide hackathon, resulting in the creation of over 800 agents—more than the number of employees—and reaching a point where over 97% of employees use AI on a daily basis. Also, at the major medical platform company Epic, management themselves built apps during meetings, which reportedly led to more than half of non-engineering employees using AI daily.
I think there is a top-down approach where management and leaders say, 'We are going to innovate using generative AI as well.' Of course, that is extremely important, but I think the bottom-up approach, where people on the front lines start using it and then continue to build it up from within, is a very important approach.
The key to not keeping this "bottom-up passion" from the front lines contained within the company, but rather connecting it with external knowledge, is the existence of a "community."
The Claude community has been held nearly 300 times in over 100 cities across 37 countries, with about 40,000 participants. On this day, recruitment also began in Japan for the "Claude Community Ambassador," who will be a leader in spreading AI to the front lines.
Mr. Tsuji's argument is extremely simple. What is most important in the coming era is not the skill of writing code, but "domain knowledge" (deep knowledge of the field in that area).. Anthropic's AI researchers do not know the struggles of law firm operations, the flow of a consultation room, or the road conditions in Uganda.
The experts on the front lines are the ones who understand the problems in that domain better than anyone else, and it was a powerful message that an era has arrived where "people who understand the issues" can build software themselves to solve them.
■ Recommendations that serve as "action guidelines for tomorrow"
Finally, based on the entire three-part report, I will state three recommendations that serve as "action guidelines for tomorrow."
First, start with "small, repetitive tasks where your domain knowledge can be utilized."
The strength of the winning lawyer was not programming ability, but knowing inside out "which part of legal work is the most troublesome." The "30 minutes you repeat the same steps every time" in your work will be the best practice ground for delegating work to AI agents.
Second, have the habit of putting into words "what constitutes a passing grade."
Rakuten's "evaluation by task unit price", NRI's "acceptable line from a business perspective", Canva's "end-to-end Evals", and Theo Chu's "including problems that cannot yet be solved in the evaluation". Everyone, regardless of their position or industry, was ultimately talking about the same thing—"have a clear definition of good work in your own words." This is a universal technique (management skill) that is effective not only for AI but also when delegating work to human subordinates and colleagues.
Third, incorporate the generational turnover of models into your schedule not as a 'response cost,' but as a 'regularly occurring opportunity.'
As the latest research from METR shows, the length of tasks that AI can autonomously complete is doubling every few months. Just as Canva rewrote its control program (harness) three times, and Rakuten redesigned its operations from scratch with every model evolution, only organizations that build systems on the premise that 'AI will quickly become smarter' are turning the next technological leap directly into business results.
Conversely, please do not discard attempts that did not go well today as failures, but keep them as 'Evals' (evaluation tests). When you run them a few months later on the day a new model is released, they will become the 'best parting gift' for you.
⭐️ Reference Bibliography
○ 'Code with Claude 2026 | Tokyo (Live Stream Archive)' - Anthropic (June 10, 2026, YouTube) The primary source upon which this article relies. Includes the lecture by Theo Chu (around 6:01), the lecture by Brad Abrams & Rod Howarth (around 7:31), and the lecture by Junichiro Tsuji (around 8:16). Basis for Chapters 6, 7, and the Final Chapter.
○ 'Measuring AI Ability to Complete Long Tasks' - Thomas Kwa et al., METR (2025, arXiv:2503.14499) Measured 13 models across approximately 170 software tasks. Reported that the 'time horizon for a 50% success rate has doubled approximately every 7 months from 2019 to 2025,' and predicted the automation of month-long tasks within 5 years via extrapolation. Basis for Chapter 6 (note that this is an extrapolation of past trends and not a guarantee of the future, and that the tasks were prepared for research purposes).METR explanation page included.
○ 'SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?' - Xiang Deng et al., Scale AI (2025, arXiv:2509.16941) A more difficult benchmark consisting of 41 repositories and 1,865 questions, with reference solutions averaging 107.4 lines and 4.1 files. Reported on a design that emphasizes data contamination prevention and practical utility. Auxiliary line for the 'saturation' discussion in Chapter 6 (the original source of the test where Fable 5 was deemed the highest performance in Part 1, Chapter 1).Public leaderboard included.
○ 'Prompt caching (Official Documentation)' - Anthropic (Official Documentation) Describes specifications such as cache reads being 10% of the base input price (effectively 90% off) and up to 85% latency reduction. Basis for Chapter 7, Prompt Caching section.Announcement at general availability included.
○ 'Introducing advanced tool use on the Claude Developer Platform' - Anthropic (Official Engineering Blog) Explains the design and effects of tool search tools and programmatic tool calls. Reported that 'average token usage decreased by 37% from 43,588 to 27,297 in complex research tasks.' Basis for Chapter 7, Context Engineering section (note that this is in-house measurement regarding their own products).Context Engineering Cookbook included.
○ 'Prompting Claude Fable 5 (Japanese Official Guide') - Anthropic (2026, Official Documentation) Official prompt guidance for bringing out the performance of the new model. Can be referenced as a practical guide for 'shortening the scaffolding' and 'writing with intent' in Chapter 6.Japanese model introduction included.
※ AI was used for the structure and preliminary research of this article.
[Editor's Note / Bonus]
By the way, regarding the thumbnail image for this article, in honor of the new model "Fable," I made a play on the "Fable" from a certain famous manga about a genius hitman. Did you notice? (laughs)
Unfortunately, it had to be taken down, but I've finally managed to get things back on track.
This has been a long report, but thank you so much for reading until the end!