(LSJ) a four-layered framework for building robust and scalable AI agent systems

/

The document presents a four-layered framework for building robust and scalable AI agent systems, moving from abstract reasoning to real-world adaptation:

  • Layer 1: The LLM (Reasoning) 🧠: The "Brain" providing core cognitive power, but isolated. (e.g., Gemini)

  • Layer 2: The Agent (Execution) 👐: The "Hands" connecting the LLM to the world using tools and grounding (RAG) for action. (e.g., Vertex AI Agent Builder)

  • Layer 3: The Orchestrator (Coordination) 🧑‍💼: The "Manager" that manages workflow, breaks complex tasks into sub-tasks, and coordinates a team of specialized agents. (e.g., LangGraph)

  • Layer 4: The Learning Loop (Adaptation) 🔄: The key to "AI Engineering 2.0," providing system-level adaptation through Persistent Memory and Feedback Loops (self-correction or human-in-the-loop). (e.g., Memory Bank)

The framework is shown through business cases (e.g., insurance claims, supply chain) and suggests new required roles for AI engineering teams.

The following post presents an expanded academic framework inspired by a sharp, layered model for AI agents originally shared by Connor Davis.

From Thinkers to Doers: A Layered Framework for AI Agents

The term "AI Agent" has become central to discussions about artificial intelligence, but its meaning can be ambiguous. To build robust and scalable AI applications, it's essential to move beyond the hype and adopt a structured framework.

A modern AI agent is not a single tool. It is a system comprised of distinct abstraction layers, each with a specific function. We can conceptualize this stack as four interconnected layers: the LLM (Reasoning), the Agent (Execution), the Orchestrator (Coordination), and the Learning Loop (Adaptation).

Layer 1: The "Brain" (The Reasoning Layer) 🧠

At the foundation of any agent is the Large Language Model (LLM), such as Google's Gemini. This is the core "brain" or reasoning engine.

  • Function: It provides the raw cognitive power. It can read, write, reason, and synthesize information based on the vast "world knowledge" it acquired during training (Vaswani et al. 2017).

  • Limitation: In its raw form, an LLM is isolated. It is non-stateful (it has no memory of past interactions) and cannot interact with the external world. It's like a brilliant mind in a locked room—all potential, no action.

Layer 2: The "Hands" (The Execution Layer) 👐

This layer gives the "brain" a "body," connecting its abstract reasoning to the real world. This is where the LLM becomes a functional agent.

  • Function: This layer provides the LLM with tools and grounding.

  • Tool Use: Through function calling, the agent can interact with external APIs. This allows it to check a database, send an email, or access real-time data.

  • Grounding: Using Retrieval-Augmented Generation (RAG), the agent can access specific, external knowledge (like your company's documents) to ground its answers in factual, private data (Lewis et al. 2020).

  • Illustrative Technology: Vertex AI Agent Builder is a prime example of this layer. It provides the platform to connect a Gemini model to data sources (via Vertex AI Search) and external tools, turning a "thinker" into a "doer."

Layer 3: The "Manager" (The Coordination Layer) 🧑‍💼

While a single agent is useful, complex problems require a team. The orchestrator acts as the "manager," coordinating multiple specialized agents.

  • Function: This layer manages workflow and decomposition. Instead of one "god" agent trying to do everything, the orchestrator breaks a complex task (e.g., "analyze our quarterly sales") into sub-tasks and routes them to a team of specialized agents:

  • Analyst Agent: Queries a BigQuery database.

  • Coder Agent: Writes Python to visualize the data.

  • QA Agent: Validates the code and the results.

  • Illustrative Technology: Open-source frameworks like LangGraph (which integrates with Vertex AI Agent Engine) are built for this. They create stateful graphs that manage the flow of information between agents, enabling complex, multi-step reasoning and action (Yao et al. 2022).

AI Engineering 2.0: The Learning Layer 🔄

The first three layers describe a powerful, static system. The final layer—and the key to AI Engineering 2.0—is adaptation. This is what makes the system dynamic, stateful, and truly "intelligent."

This "learning" does not mean re-training the base LLM. It refers to system-level adaptation through two key mechanisms:

  1. Persistent Memory: The agent must be stateful, remembering key details from past interactions. Services like the Vertex AI Agent Engine Memory Bank are designed for this, allowing an agent to build a long-term understanding of a user's preferences and context.

  2. Feedback Loops: The system must learn from its mistakes in real-time. This can be autonomous or human-assisted.

  • Self-Correction: A common pattern in orchestration is the "Generator-Critic" loop. One agent (the generator) produces work, and another agent (the critic) evaluates it based on a set of rules, forcing an iterative refinement (Bai et al. 2022).

  • Human-in-the-Loop: In collaborative tools like the Cursor.ai editor, the human developer acts as the manager/critic. The developer prompts the AI, reviews its code, and provides immediate feedback, creating a tight, collaborative learning loop.

Building the Team: Roles for AI Engineering 2.0

This new stack requires new roles that bridge the gap between data science, software engineering, and business operations.

  • AI Architect: This is the high-level designer. They map the business problem to the four-layer stack, select the right models (e.g., Gemini), design the data pipelines, and establish governance and security frameworks.

  • Knowledge Architect: This role focuses on the "Brain" and "Hands" (Layers 1 & 2). Their duty is to curate the agent's knowledge. They manage the RAG data sources, define the agent's persona, and ensure its skills and context are accurate and aligned with the company's voice.

  • Orchestration Engineer: This is a specialized developer who works at the "Manager" layer (Layer 3). Their duty is to build and manage the workflows in tools like LangGraph. They design the logic, conditional branches, and error handling that allow multiple agents to collaborate effectively.

  • AI Operations Manager (or "Agent Fleet Commander"): This role focuses on the "Learning Loop" (Layer 4). Once the agent team is deployed, this person manages its performance. Their duties include monitoring for failures, reviewing agent-to-agent interactions, flagging incidents for review, and overseeing the "human-in-the-loop" feedback process.

Business Cases for the Full Stack 📈

This four-layer framework isn't just a theory; it's actively being deployed to solve complex, real-world problems that a simple chatbot could never handle.

  • Automated Insurance Claims Processing: This is a classic long-running process.

  • Layer 1 (Brain): Gemini understands the user's claim description ("A tree fell on my car").

  • Layer 2 (Hands): An Intake Agent uses tools to create a case file. A Policy Agent checks the user's policy status via an API.

  • Layer 3 (Manager): An Orchestrator (e.g., LangGraph) manages the workflow. It routes the case to an Eligibility Agent. If the claim is complex, it triggers a Human-in-the-Loop step, assigning it to a human adjuster.

  • Layer 4 (Loop): A Fraud Detection Agent runs in parallel, flagging unusual patterns. The system "learns" from the human adjuster's decision, improving its routing for future claims.

  • Dynamic Supply Chain Management: In e-commerce, inventory is a complex, multi-agent problem.

  • Layer 1 (Brain): A reasoning model can interpret unstructured signals, like a news report about a port closure.

  • Layer 2 (Hands): A Demand Agent monitors sales data. A Logistics Agent tracks shipments via API.

  • Layer 3 (Manager): An Orchestrator coordinates the team. When the Demand Agent forecasts a shortage, the manager tasks a Procurement Agent to automatically generate a purchase order with a supplier.

  • Layer 4 (Loop): The system continuously learns, adjusting its "days of supply" thresholds based on the real-world performance (e.g., lead times, sales velocity) of its agent team.

  • Proactive Customer Support: Moving beyond reactive chatbots.

  • Layer 1 (Brain): A model can understand a frustrated user's email.

  • Layer 2 (Hands): A Triage Agent identifies the user's intent. A Billing Agent checks their account status. An Authentication Agent verifies their identity.

  • Layer 3 (Manager): An Orchestrator receives the case. If the Billing Agent finds a simple overcharge, the Orchestrator empowers a Refund Agent to automatically process the refund and close the ticket—no human intervention required.

  • Layer 4 (Loop): A Memory Bank (Layer 4) remembers this interaction. The next time the user contacts support, the agent already has the context of this past issue.

A Modern AI Stack

This framework helps clarify how these components fit together to create sophisticated applications.

Layer

Core Analogy

Core Function

Illustrative Technologies

Layer 1

The Brain

Thinks & Reasons

Gemini

Layer 2

The Hands

Does & Acts

Vertex AI Agent Builder (using Search & API tools)

Layer 3

The Manager

Coordinates & Manages

LangGraph, Cursor.ai (with human-in-the-loop)

Layer 4: the Process

The Loop

Adapts & Learns

Memory Bank & Self-Correcting Graphs

Further Learning & Resources 📚

For those interested in building systems based on this framework, Google offers several courses and learning paths:

References

  • Bai, Y., et al. (2022). 'Constitutional AI: Harmlessness from AI Feedback'. arXiv preprint arXiv:2212.08073.

  • Davis, C. (2024). X post. [Online]. Available: https://x.com/connordavis_ai/status/1985998034506060012?s=46 [Accessed 6 November 2025].

  • Lewis, P., et al. (2020). 'Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks'. Advances in Neural Information Processing Systems 33 (NeurIPS 2020).

  • Vaswani, A., et al. (2017). 'Attention Is All You Need'. Advances in Neural Information Processing Systems 30 (NIPS 2017).

  • Yao, S., et al. (2022). 'ReAct: Synergizing Reasoning and Acting in Language Models'. arXiv preprint arXiv:2210.03629