In the Era of AI Agents, Will Infrastructure Technology Become Obsolete? — The Design Philosophy of Next-Generation Control Foundations (Harnesses) to Overcome the 'Three Walls'

From 2025 to 2026, a strange reversal has been occurring in the field of enterprise AI.

Excellent infrastructure engineers who have optimized network routing, strictly normalized RDBs, and designed multi-layered security architectures suddenly stop in their tracks when it comes to integrating LLMs into business systems. They freeze in front of their screens. They lose their design compass.

This is not a problem of the technology stack, such as "which API to use" or "which framework to choose."

It stems from a deeper disconnect—the fact that the very paradigm that serves as the premise for the system has been turned 180 degrees upside down.

And for companies that rush to implement AI without understanding that disconnect, a certain cold data awaits.

Introduction: Why 88% Never Reach 'Production'

Market data is cruel. 88% of enterprise AI agent projects end without ever reaching a production environment. This figure is not because the models lack capability.

The 88% figure has not improved, no matter how much smarter the models have become. This is because the improvement of capability and the integration into production sites are entirely different problems.

65% of the cause lies not in the model itself, but in defects in the 'Harness'—that is, design flaws in the control foundation surrounding the AI. Context drift, schema misalignment, and state degradation are the primary root causes of enterprise AI failure.

Moreover, ironically, in a randomized controlled trial by METR (Model Evaluation & Threat Research) published in July 2025, the group that used AI tools took 19% longer to complete tasks than the group that did not.

Even though developers predicted in advance that they would be '24% faster with AI' and subjectively evaluated that they were '20% faster' even after completing the tasks, the actual measured values were the opposite.

Why? Because there was no 'harness' to verify and control the AI's output, humans had to manually review code, adjust prompts, and perform tests—in other words, humans themselves were forced to function as a 'manual harness'.

The key to understanding this paradox lies in the 'three paradigm shifts'.

Paradigm Disconnect: Determinism vs. Probabilism

Conventional enterprise systems have been built with a deterministic design philosophy. The same input always returns the same output. Covering all exception handling, bringing bugs to zero, and keeping the system completely under human control were the supreme mandates for engineers.

In contrast, AI systems centered on LLMs are inherently probabilistic. Even with the same input, fluctuations occur in the output. To incorporate these non-deterministic components into mission-critical operations, a fundamental update to one's mental model is required.

Engineering for AI system construction has gone through three generations from 2022 to 2026.

• First Generation (2022–2024): The Era of Prompt Engineering — Attempting to control LLMs by refining single instructions.

• Second Generation (2025): The Era of Context Engineering — The focus shifted to dynamically constructing the context referenced by LLMs using tools like RAG.

• Third Generation (2026–): The Era of Harness Engineering — Designing the entire environment surrounding AI models (tools, permissions, sandboxes, evaluation loops) has become the source of competitive advantage.

In this article, we will systematically dissect the 'three walls' corresponding to these three generations. We will then discuss, along with the latest research and case studies, why traditional infrastructure technology is becoming more important than ever as 'steel chains to safely tame AI'.

The First Wall: The Wall of Prompt Engineering

'From Code to Natural Language' — From Absolute Testing to Statistical and Semantic Testing

Prompts are not 'code'

In traditional software development, interfaces and logic are described using GUIs, CLIs, or strict If-Then-Else conditional branching. For a given input, the output is 100% deterministic, and minimizing exceptions was the aesthetic of an engineer.

However, the LLM interface is Natural Language (LUI: Language User Interface). Due to the Temperature parameter and probabilistic reasoning, the output fluctuates even with the same prompt. Whether you view this as a 'bug' or a 'characteristic to be managed' changes an engineer's design philosophy by 180 degrees.

There is an even harsher reality. The approach of trying to solve this problem through fine-tuning prompts—prompt engineering—is already reaching its limits. According to research by the Stanford University HAI group in late 2025, it has been demonstrated that the improvement in output quality brought about by prompt optimization for advanced models remains less than 3%.

Reinventing TDD for LLMs

The shift required here is to let go of the mindset of 'completely controlling the system.' What is needed instead is the 'design of excellent evaluation metrics' and the 'installation of guardrails (output control)'—this is what the application of Test-Driven Development (TDD) to LLMs achieves.

However, TDD for LLMs is different from the traditional kind. LLMs do not fully grasp the intent of instructions, and output formats are prone to fluctuation. Quality cannot be guaranteed by exact string matching or simple assertions (such as character count checks) alone.

Practical design guidelines are as follows.

• First, build a 'Golden Dataset' that defines expected diverse inputs and ideal behaviors.

• Incorporate retry logic into the test suite and introduce the concept of a 'Global Pass Rate' (e.g., 80% success within a certain number of trials).

• Accepting the premise that LLMs fail probabilistically, define the acceptable range of deviation with data.

Evolution of Evaluation Metrics: From BLEU/ROUGE to LLM-as-a-Judge

Selecting metrics to measure LLM output quality is at the core of system design. Traditional statistical scorers like BLEU, ROUGE, and METEOR mathematically compare N-gram matches between output text and ground truth text. While fast and deterministic, they have the fatal flaw of being unable to capture the 'semantic nuances' or 'validity of logical reasoning' in text generated by LLMs.

Overcoming this limitation is 'LLM-as-a-Judge'. This involves having another LLM with powerful reasoning capabilities evaluate the target output based on natural language scoring criteria (rubrics).

Currently, the following evaluation frameworks are frequently used in practice.

• DeepEval, RAGAS, TruLens — Cutting-edge open-source evaluation frameworks centered on LLM-as-a-Judge.

• G-Eval — A framework that utilizes Chain-of-Thought to evaluate outputs based on arbitrary custom criteria. It allows for defining complex requirements in natural language—such as "Is the output factually consistent with the provided context (Faithfulness)?", "Does it appropriately reject malicious inputs?", or "Does the output tone comply with corporate guidelines?"—and obtaining them as normalized scores from 0 to 1.

• RAGAS (Answer Relevancy, Faithfulness, etc.) — A specialized tool for evaluating the retrieval accuracy and generation quality of RAG architectures based on relevance and faithfulness to the context.

When organizing evaluation methods by purpose, they can be used as follows:

• Baseline evaluation for machine translation or strict format matching → Statistical scorers (BLEU, ROUGE)

• When high-speed operation is required for hallucination detection or toxicity filtering → Model-based classifiers (NLI, BLEURT)

• Evaluating tone appropriateness, reasoning processes, or regression testing during prompt changes → General-purpose LLM-as-a-Judge (G-Eval)

• Specialized evaluation for RAG system retrieval accuracy and hallucination detection → RAGAS

The shift required of engineers is clear.From "eliminating output fluctuations as bugs" to "defining acceptable deviation ranges with data and building guardrails for continuous monitoring within the CI pipeline"—this shift in thinking is the key to overcoming the first wall.

The Second Wall: The Wall of Context Engineering and Data Defense

From "Exact Match" to "Semantic Similarity Scoring," and then to Dynamic Access Control

The "New Common Sense" of Vector Databases

For traditional engineers, searching internal data meant RDB (Relational Database) and SQL. Performing exact match searches on structured data using IDs or keywords and returning accurate results in milliseconds—this is the premise that "this is what data is."

However, in RAG (Retrieval-Augmented Generation) architectures designed to help LLMs understand unstructured data such as internal regulations, contracts, and technical manuals, the protagonist is entirely different. The focus shifts to vector databases (Pinecone, ChromaDB, Elasticsearch, etc.) that convert data into directions and magnitudes in a multi-dimensional vector space and perform fuzzy searches based on mathematical similarity (cosine similarity).

From "exact match" to "semantic relevance scoring"—if you proceed with implementation without understanding this conceptual shift, you will never find the answer to the question of "why are the search results not accurate?"

RAG Security: The "Fatal Hole" Often Overlooked

However, migrating to a vector DB only solves half the problem. When a RAG system accesses enterprise confidential data, security risks of unprecedented scale emerge.

LLMs themselves lack the ability to determine which user can access which data. Therefore, if access control is not strictly enforced in the search pipeline, confidential information for management or personal information (PII) may be retrieved as relevant context for a general employee's query and output as an answer—this is "data exfiltration."

In the OWASP Top 10 risks for LLM applications, "Sensitive Information Disclosure" and "Vector and Embedding Weaknesses" are listed at the top.

Dynamic Access Control: From RBAC to ABAC/ReBAC

The key to solving this challenge is to integrate the expertise in robust access management that is the exclusive domain of traditional engineers into the vector search process.

The simplest approach is "metadata filtering," where static role-based access control (RBAC) tags (e.g., allowed_groups: ["finance"]) are assigned to each vector chunk during the ingestion phase. This is easy to implement and has low latency.

However, in an enterprise setting, this hits limits in terms of scalability and real-time performance. Because permissions fluctuate dynamically due to personnel changes or project shifts, synchronization lag occurs between the static metadata in the vector DB and identity providers (IdP) like Active Directory. During this lag period, a security hole is created where a user whose permissions have been revoked can still search for sensitive data.

Modern enterprise RAG systems employ more advanced approaches.

• ABAC (Attribute-Based Access Control) — Authorizes access by evaluating not just a user's job title, but also dynamic context such as access location, time of day, and the security state of the device.

• ReBAC (Relationship-Based Access Control) — Solutions like SpiceDB, inspired by the Google Zanzibar paper, calculate permissions based on complex graph relationships between users and resources, injecting them as filtering conditions in real-time during vector search query execution.

• Open Policy Agent (OPA) / AWS Cedar — A policy engine for decoupling complex authorization logic from application code. By combining Cedar with control planes like OPAL, microsecond-level permission checks can be achieved at scale.

In architectures like Amazon Bedrock Knowledge Bases, the moment a user issues a query, an authentication token is received from an identity provider like Amazon Cognito, and a Lambda function extracts the user's claims (permission information). Then, a vector search is executed targeting only the context that matches the user's attributes, physically preventing unauthorized data from being mixed into the LLM prompt.

Another essential practice from a zero-trust perspective is PII sanitization during ingestion. Highly confidential data that does not need to be read by the LLM, as well as PII such as social security numbers and credit card information, should be masked or redacted before being stored in the vector DB. By using tools like Amazon Comprehend to replace PII with placeholders, it becomes a cornerstone of defense-in-depth, preventing the leakage of sensitive values even if there is a flaw in the index access control.

The core of overcoming the second wall is not to "lament the uncertain behavior of LLMs," but to establish deterministic and robust access control checkpoints in the "data pipeline" and at the "vector DB entrance" long before the query reaches the LLM.

The Third Wall: The Wall of the Boundary Between Harness Engineering and Autonomy

How to Design the "Reins" Surrounding AI

What is Happening on the Front Lines in 2026

The final and greatest wall in building AI systems is the shift in the infrastructure and security design paradigm that governs the entire architecture.

Traditional infrastructure engineers were professionals who built "robust castles" by designing strict static routing using VPCs, firewalls, and IAM to prevent external threats at the perimeter. However, as AI evolves from a "machine that returns text" into an "autonomous AI agent" that performs reasoning, plans autonomously, and calls external APIs or databases to execute tasks, traditional static perimeter defense is no longer sufficient to protect the system.

As of 2026, a discipline called "Harness Engineering" is being established at the forefront of AI development.

A discipline named Harness Engineering was born in early 2026. The trigger was an announcement regarding Agent infrastructure by OpenAI engineer Ryan Lopopolo. A few days later, Mitchell Hashimoto, the creator of Terraform and Ghostty, distilled that core insight into a formula that practitioners could immediately use.

That formula is—

.

A harness refers to the entire external infrastructure used to enclose a non-deterministic engine like an LLM, controlling and monitoring its actions through deterministic rules.

The model provides pure cognitive reasoning as a stateless token predictor, while the harness constitutes the runtime software infrastructure that coordinates tool dispatch, context management, and safety enforcement.

A map of control drawn by Thoughtworks engineers

Thoughtworks senior engineer and AI-assisted development expert Birgitta Böckeler analyzes harnesses in detail on Martin Fowler's blog as a two-layer structure of "feed-forward control (Guides)" and "feedback control (Sensors)."

Feed-forward control (Guides) is a mechanism that explicitly defines rules, boundaries, and architectural conventions before an agent takes action, limiting the action space to a safe domain.

• Machine-readable specifications (AGENTS.md) placed within the repository

• Strict JSON schema definitions accepted by tools

• Fitness functions that define architectural requirements

Feedback control (Sensors) is a mechanism that verifies whether the results meet system requirements after an agent takes action, prompting the agent to self-repair if errors occur.

This verification mechanism should be multi-layered. Deterministic and high-speed "Computational Controls"—such as linters, static analysis, and unit tests—should be placed at the forefront, and only those that pass should be passed on to non-deterministic and costly "Inferential Controls," such as semantic reviews by LLM-as-a-Judge.

Computational sensors reliably detect structural issues—duplicate code, cyclomatic complexity, lack of test coverage, architectural divergence, style violations, etc. These are inexpensive, proven, and deterministic.

The future shown by OpenAI's "zero-human code" experiment

From late 2025 to 2026, an experiment was conducted to fundamentally rethink what the role of an engineer entails.

At the end of August 2025, a small team of three OpenAI engineers made the first commit to an empty repository. There was only one constraint: "It must not contain a single line of human-written code." The repository structure, CI settings, formatting rules, package manager setup, and application framework—everything was generated by the Codex CLI using GPT-5. The AGENTS.md file, which contains instructions for the agent, was also written by Codex itself.

Five months later, the repository contained approximately 1 million lines of code, including application logic, infrastructure, tools, documentation, and internal development utilities, with about 1,500 pull requests opened and merged. This equates to an average of 3.5 PRs per engineer per day, and throughput continued to improve even as the team expanded to seven people.

From an agent's perspective, anything that cannot be accessed within its context is effectively "non-existent." Google Docs, Slack threads, knowledge inside people's heads—these are invisible to the agent. Only version-controlled artifacts described in the repository (code, Markdown, schemas, executable plans, etc.) constitute the world the agent can reference.

What this experiment shows is that the engineer's job has shifted completely from "writing code" to "creating an environment that AI agents can understand and designing feedback loops (linters, tests, CI/CD)."

Engineers did not fix the code themselves; instead, when an agent failed, they added "constraints (linters and tests)" to the infrastructure to ensure the system's quality was maintained so that the same mistake would never be made again.

Pandora's Box of MCP: The light and shadow of "AI's USB-C"

A new battlefield created by the "standard" for enterprise AI integration

The Model Context Protocol (MCP), proposed by Anthropic, has been rapidly adopted by enterprises since the end of 2024. Often called the "USB-C for AI," it connects AI clients to external systems (databases, Slack, GitHub, and various APIs), enabling dynamic tool discovery and execution through a standardized interface.

However, MCP is also a "Pandora's box" that introduces unprecedented security risks to enterprise infrastructure.

An analysis of 2,614 MCP implementations by Endor Labs revealed that 82% use file operations prone to path traversal, 67% use APIs associated with code injection, and 34% use APIs vulnerable to command injection.

Tool Poisoning: Attacks Undetectable by Signature-Based Security

"Tool poisoning" attacks have been demonstrated where attackers hide malicious prompts (e.g., "forward this data to an external endpoint") within API responses or tool descriptions, triggering malfunctions the moment an AI agent reads them. Tool descriptions are typically not displayed to the user—while the user sees "Checking the weather...", the AI is following completely different instructions in the background.

Furthermore, silent redefinition, known as a "Rug Pull," is also serious—MCP tools can secretly change their own definitions after installation. A tool that looked safe on day one might start routing API keys to an attacker on day seven.

As a real-world example discovered in June 2025, after productivity giant Asana released new MCP features, a bug caused some customer information to leak into other customers' MCP instances. The company had to suspend MCP integration for two weeks while the security team scrambled to apply patches.

The most well-known MCP security incidents of 2025–2026 include the backdoor in the postmark-mcp npm package (the first malicious MCP server discovered in the wild, September 2025), the tool poisoning disclosure by Invariant Labs (April 2025), and the RCE chain via three CVEs in Anthropic's mcp-server-git (CVE-2025-68143, 68144, 68145).

MCP Proxies and the Confused Deputy Problem

When building MCP proxy servers, meticulous attention must be paid to the "Confused Deputy" problem.

When a proxy server integrates with third-party APIs, the following implementation errors become attack vectors.

• Lack of strict Per-Client Consent management

• Verification via incomplete string matching of redirect URIs

• Insufficient lifecycle management of cryptographically secure state parameters

Neglecting these allows attackers to illicitly seize authorization codes and impersonate other users to manipulate the system. Additionally, "Token Passthrough, where tokens received from a client are passed directly to downstream APIs, is an anti-pattern explicitly prohibited by the MCP implementation specification because it leads to a lack of audit trails and the bypassing of security controls.

Sandboxing and Zero Trust: The "Physical Walls" Imposed on Autonomous AI

Why Docker Is Not Enough

The ultimate "harness" for controlling the uncertain behavior of autonomous AI agents and the risks of external tool execution via MCP is the construction of highly advanced sandbox environments.

Here, a shift in perception that overturns the common sense of traditional engineers is required.

Standard container technologies like Docker share the host OS kernel. This is fundamentally insufficient as an isolation barrier for executing unverified AI-generated code. If a kernel vulnerability is exploited via prompt injection or similar means, container technology cannot prevent the damage from spreading to the host infrastructure or other tenants.

The recommended architecture for AI code execution in the enterprise is as follows.

• MicroVMs (Firecracker, Kata Containers) — MicroVM technology like Firecracker, which is also used by AWS Lambda, achieves hardware-level isolation using KVM. Each instance is assigned a dedicated lightweight kernel, providing the highest level of security. Modern sandbox platforms for AI agents, such as E2B and Modal, adopt this technology and use an ephemeral design that discards state after code execution.

• Userspace Kernel (gVisor) — Intercepts system calls to the host kernel and provides independent kernel emulation. While not full hardware isolation, it is effective for processing trusted code.

• Standard Containers (Docker/OCI) — Shares the host OS kernel and is limited to logical isolation via Namespaces and cgroups. It is unsuitable for executing unverified LLM-generated code and carries a risk of privilege escalation.

Zero Trust and Human-in-the-Loop

Applying a zero-trust architecture across the entire system is also essential. AI-generated outputs and requests, even if generated by trusted agents within the system, must be treated as completely "Untrusted Input". Always deploy a gateway (such as an MCP gateway) to sanitize and verify all parameters before execution.

Regarding authorization, strictly adhere to the Principle of Least Privilege. Strictly separate read-only tools from writable tools, and for high-risk, destructive operations (such as database updates, payment processing, or sending external emails), it is essential to design a process that incorporates human approval (Human-in-the-Loop).

Conclusion: "Steel Chains" to Tame the "Probabilistic Engine"

The three walls that traditional IT engineers face when building AI systems are—

1. Uncertainty in evaluating natural language interfaces (The wall of prompt engineering)

2. Integration of vector search and dynamic data defense (The wall of context engineering)

3. Infrastructure design to control and isolate autonomous agents (The wall of harness engineering)

These do not mean that existing IT infrastructure skills have become obsolete. In fact, the opposite is true.

"The harness determines the success or failure of an AI product" is now a common understanding in AI product development. Even if two products use the same base LLM, the one with a harness that has superior tool support, memory, and user guidance will provide an overwhelmingly better user experience.

The most important discovery in recent deployments is that a lack of model reasoning capability is rarely the cause of project failure. 65% of enterprise AI failures are caused by defects in the harness—context drift, schema mismatches, and state degradation. Even if you optimize the model, if you do not stabilize the harness, returns will continue to diminish.

And this is exactly where traditional infrastructure engineers shine.

Network micro-segmentation, robust authentication and dynamic authorization (utilizing OPA/Cedar, etc.), vector database architecture design, and secure microVM technology (Firecracker, etc.)—these "deterministic and rigid guardrails" are absolutely essential for implementing the "probabilistic engine" that is the LLM to a level that can withstand mission-critical enterprise operations.

Models are replaceable components. The harness is the product.

When infrastructure engineers deeply understand the probabilistic premise that "AI output is inherently volatile" and can redefine the deterministic infrastructure technology, test-driven philosophy, and zero-trust mindset they have cultivated over many years as a "robust foundation (harness) to safely tame AI"—

the organization's AI implementation will move beyond experimental PoCs and make a dramatic leap into an enterprise system that is truly scalable and creates business value.

The person who designs an "extremely safe, isolated box (sandbox)" where AI can think and experiment freely—the protagonist of the next era might be you.

List of Information Sources

Key Papers and Research

• Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (METR, arXiv:2507.09089) — An RCT study showing experienced developers were 19% slower when using AI tools

• METR Official Blog (July 10, 2025)

• METR Official Blog (Updated February 24, 2026)

• Agent Harness for Large Language Model Agents: A Survey (Preprints.org)

• Architectural Design Decisions in AI Agent Harnesses (arXiv:2604.18071)

• Model Context Protocol Threat Modeling (arXiv:2603.22489)

• Prompt Injection Attacks in LLMs (MDPI, January 2026)

Industry Articles and Blogs

• Harness engineering: leveraging Codex in an agent-first world (OpenAI Official) — Official commentary on OpenAI's "zero-human code, 1 million lines" experiment

• Harness engineering for coding agent users (Martin Fowler Blog, Birgitta Böckeler)

• Agent Harness Engineering — The Rise of the AI Control Plane (Medium, Adnan Masood)

• What Is Harness Engineering AI? The Definitive 2026 Guide (Atlan)

• The Agentic Harness: Why the Orchestration Layer Is the Product (Veso AI)

• What is an agent harness in the context of LLMs?（Parallel AI）

• Harness Engineering: The Complete Guide（NxCode、2026）

• OpenAI Extreme Harness Engineering（ZenML LLMOps Database）

• Engineering for Agents: Building a Million-Line Codebase（Tecyfy）

Security-related

• OWASP MCP Top 10（Official）

• MCP Security Vulnerabilities（Practical DevSecOps）

• The State of MCP Security 2026（PipeLab）

• Model Context Protocol has prompt injection security problems（Simon Willison）

• MCP Vulnerabilities Every Developer Should Know（Composio）

• Top 10 MCP Security Risks（Prompt Security）

• New Prompt Injection Attack Vectors Through MCP Sampling（Palo Alto Unit 42）

• Top MCP Security Resources — February 2026（Adversa AI）

OSS Projects & Tools

• GitHub Awesome Harness Engineering

• OpenHarness (HKUDS)

• Harness Agents (Harness Developer Hub)

Developer Experience and Productivity

• AI Coding Tools Made Developers 19% Slower: METR Study (Let's Data Science)

• METR's AI productivity study is really good (Sean Goedecke)

• The Architect and the Builders (LawDroid Manifesto)

• OpenAI Distills Engineers' Experience into Skills (36Kr)

Please also refer to the following for your reference.
https://note.com/betaitohuman/n/ndab885850898

Tags

#HarnessEngineering #ITEngineers #AIInfrastructure #AccessControl #DevOps #AIProducts

If you are interested, please click the Like❤️ button. It will also serve as a bookmark. If there are any other themes you are interested in, please let me know even just the keywords, and I will use them as material for future topics.