Blog

Agentic AI Is No Longer a Research Problem - The Question Now Is Whether You're Building It Right

Our Read on the Emerging Agentic AI Stack and Why It Matters Now

Inspired by NVIDIA GTC 2026

14 May 2026

NVIDIA's GTC has always been a bellwether for what's coming in AI. But NVIDIA GTC 2026 felt different. Members of the Office of CTO at Amdocs Studios each worked through their own queue, sessions recommended by peers, bookmarked from areas we're actively building in, and then we started comparing notes.

What emerged from those conversations surprised us. Independently, we had each landed on the same observation: this year was not a collection of isolated announcements. The more we talked through what we'd each seen, the clearer it became that the sessions, taken together, painted a coherent picture of a complete, production-ready agentic AI stack, from raw inference infrastructure all the way up to enterprise deployment and security hardening.

This post is the result of that collaborative read. We’ve woven together the sessions we found most relevant, along with additional research we did on the side, into a single narrative from our own vantage point, with a natural NVIDIA flavor, given that’s where these conversations started. That said, the patterns and principles we kept coming back to felt bigger than any one vendor’s portfolio. The stack layers we’ll walk through – inference infrastructure, harness engineering, RL-based training, and security hardening – are, we believe, relevant to anyone building serious agentic AI, whatever platform they’re working on.

The diagram below illustrates how the technologies in each layers relate to one another unified stack. Read it from the bottom up: NVIDIA Dynamo anchors the foundation as the raw inference engine, NVIDIA OpenShell sits above it as the secure execution boundary for tool use, the cognitive layer (Context Engineering and Long-Horizon Agents) operates in the middle – managing what agents think with and how far they can plan – and NVIDIA Blueprints wraps the whole thing as the enterprise deployment surface. Garak, shown as the outer dashed boundary, represents the security testing discipline that must run continuously across every layer.

Layer 1 - Infrastructure: NVIDIA Dynamo

Every intelligent agent, no matter how sophisticated its reasoning, ultimately needs to make inference calls. NVIDIA introduced Dynamo as an open-source, low-latency distributed inference serving framework designed to scale generative AI and reasoning model inference across multi-node GPU environments at the lowest token cost. Dynamo offers a new modular architecture aimed at large-scale, multi-node GenAI serving

A central idea in Dynamo is disaggregated serving: treating prefill (prompt processing) and decode (token generation) as distinct phases with different hardware profiles. NVIDIA notes that prefill is compute-bound while decode is memory-bound; separating them onto different GPUs or nodes allows each phase to be optimized and scaled independently rather than bottlenecking on a single device.

This matters for agentic and multi-turn workloads because prompts often reuse substantial shared context (system prompts, repeated prefixes, and conversation history). Dynamo’s KV-cache-aware routing tracks cached blocks across the cluster and routes requests based on cache overlap to reduce expensive KV cache recomputation, improving latency and GPU efficiency in repeated-request patterns.

Dynamo’s core capabilities include multi-node orchestration, disaggregated prefill/decode serving, KV-cache-aware routing, and dynamic GPU planning/scheduling - all aimed at improving throughput and lowering cost for production-scale inference.

Layer 2 - Secure Access: NVIDIA OpenShell

Fast inference at scale solves the performance problem, but it also means agents can act faster, at greater reach, with less human oversight at each step, which makes controlling what they are actually permitted to do more critical, not less. Giving agents tools is essential but giving them tools without guardrails is risky - especially as agents become capable of long-horizon actions that touch files, networks, and external systems. OpenShell, highlighted at GTC 2026 as part of NVIDIA’s Agent Toolkit and released as open source, is a secure runtime designed to run autonomous agents inside isolated sandboxes and to sit “between your agent and your infrastructure,” governing what the agent can see and do and where its inference goes.

Rather than relying on “behavioral prompts” to keep an agent safe, OpenShell enforces constraints out-of-process through a policy system, so a compromised agent can’t simply talk its way around restrictions. The isolation is not just conceptual: NVIDIA describes OpenShell as providing sandboxed environments with kernel-level isolation, and the policy model explicitly uses kernel enforcement mechanisms (notably Landlock for filesystem boundaries and seccomp filtering for dangerous system calls).

What makes OpenShell especially developer- and operator-friendly is that its security posture is defined as declarative policy-as-code in YAML. A single policy file is structured into static sections- filesystem policy, landlock, and process - that are locked at sandbox creation, plus a dynamic section-network policies - that can be hot-reloaded on a running sandbox. Filesystem permissions are expressed as explicit allowlists of paths the agent can read vs. read/write (anything not listed is inaccessible), and those filesystem rules are enforced at the kernel level via Landlock LSM. Process constraints are also part of the policy model (e.g., the agent runs as an unprivileged user/group), and OpenShell applies seccomp filters to block dangerous system calls - so even if the agent generates malicious code, the OS-level execution boundary remains in force.

Network access follows a deny-by-default posture: outbound connections are intercepted and only allowed when they match policy, and OpenShell can enforce network permissions at a fine-grained level (including HTTP method/path controls for REST endpoints when enabled). When a policy blocks an action, the failure is surfaced clearly, and the system records structured deny logs that include the destination, the binary that attempted the connection, and the reason for denial. Operators can also retrieve the current policy as YAML and inspect policy revision history via CLI commands, reinforcing the “reviewable and auditable” nature of the permission model.

Design philosophy: OpenShell is not trying to prevent agents from being powerful - it is trying to make their power legible and auditable to the humans who deploy them.

Layer 3 - Harness Engineering: What Agents Think With

OpenShell defines the boundaries of what an agent can touch, but within those boundaries, the quality of what the agent actually does comes down to something harder to enforce with policy: the design of the environment the model itself reasons within. Harrison Chase, CEO of LangChain, has argued that the field has matured past "prompt engineering" and even past "context engineering" to something more precise: harness engineering, the discipline of designing everything that wraps around an LLM call. The harness is the system prompt, the tool definitions, the memory retrieval logic, the output parsers, the routing rules, and the evaluation loops. Context engineering is one layer within it.

The core insight is this: many agent failures are not model failures. They are harness failures. An agent given the wrong information, too much irrelevant information, or a growing context window that was never pruned will underperform regardless of how capable its underlying model is. The session by CodeRabbit illustrated this concretely: a significant share of their token budget goes into context preparation, not into the review itself. The reasoning model is, in a sense, the easy part. Engineering the right harness around it is the hard problem.

CodeRabbit's approach centers on assembling high-signal context before generation: indexing past pull requests and issues, applying coding guidelines and path/AST-based rules, and using call-graph techniques like "outside-diff impact slicing" to include relevant callers and callees, because many important bugs sit just outside the changed lines. This is context engineering operating inside a broader harness.

The session's practical lesson was that more context is not always better; irrelevant context dilutes signal. That aligns with the well-documented "lost in the middle" effect, making selection, ordering, and packaging of context a harness engineering problem, not a prompt-writing trick.

As Harrison Chase frames it, iteration happens at the harness, not the model. When an agent underperforms, the first diagnostic questions are harness questions: was retrieval selecting the right documents, were tool definitions precise, was the output parser recovering gracefully? In the stack view, harness engineering sits between raw inference infrastructure and business logic, a deliberate preparation step that gathers, filters, and packages what the model sees, and the designed interface through which a capable model becomes a reliable agent.

Layer 4 - Task Execution: Long-Horizon Agents

Many common agent demos and benchmarks emphasize short-horizon tasks: answer a question, fill a form, write a function. Real-world enterprise tasks look nothing like this. They involve multi-step plans that span hours, require invoking dozens of tools, produce intermediate artifacts, and must survive interruptions and errors along the way.

LangChain's GTC session presented their Deep Agents framework (github.com/langchain-ai/deepagents), specifically engineered for long-horizon task execution. Three architectural ideas separate this from conventional agent loops:

Planning tools: The agent is given explicit planning capabilities - not just a system prompt that says 'think step by step,' but structured tool calls that allow it to decompose tasks into sub-plans, assign priorities, and revise plans dynamically when circumstances change.
Subagent spawning: Complex tasks can be delegated to specialized subagents. The orchestrator agent spawns workers with narrower scopes - a web researcher, a code writer, a data formatter - each operating independently within defined constraints and reporting back structured results.

File-system memory offloading: For tasks that exceed a single context window, intermediate state - notes, drafts, partial results - is serialized to the file system and reloaded on demand. This allows agents to work on projects that would be impossible to hold in memory at once.

Together, these primitives make agents that can take on the kind of work a junior analyst or engineer might do over an afternoon - not just a single clever API call.

Layer 5 - Enterprise Deployment: NVIDIA AI-Q Blueprint

Once you have fast inference, curated context, long-horizon planning, and sandboxed tool execution, the question becomes: how do you package all of that into something an enterprise knowledge worker can actually query? The answer is the NVIDIA AI-Q Blueprint - pronounced "IQ" - an open-source, fully customizable enterprise research agent built on LangChain DeepAgents and the NVIDIA NeMo™ Agent Toolkit.

AI-Q is not an abstract reference architecture. It is a working system designed to answer the hardest version of the enterprise knowledge retrieval problem: a single query that might need a quick cited answer, or might need a multi-phase research investigation with planning, iteration, and a verified bibliography - and the system should figure out which is required without the user specifying it upfront. It currently tops both the DeepResearch Bench and DeepResearch Bench II leaderboards.

The architecture that enables this is a LangGraph-based state machine with four composable agents, each of which can run standalone or as part of the full pipeline. Every query enters through an Intent Classifier - a single LLM call that determines in one step whether the request is a direct meta-response, a shallow lookup, or a deep research task. The routing matters because the cost profile of each path is very different:

The Shallow Researcher handles bounded, tool-augmented queries and returns results in 30 to 60 seconds. The Deep Researcher runs a multi-phase loop - a planner subagent breaks the problem down, a researcher subagent iterates - and takes 2 to 10 minutes for comprehensive, report-style output. Critically, before deep research begins, a Clarifier agent engages the user in a human-in-the-loop step: it surfaces the proposed research plan and waits for approval. In enterprise settings where a misunderstood query could produce a ten-minute report on the wrong topic, this gate is not overhead - it is exactly the right design.

Citation integrity is handled by a deterministic post-processing pipeline that runs on every response, shallow or deep. It verifies each citation against the sources actually retrieved during that run, strips unverifiable or unsafe URLs, and produces an audit trail. This is always on and not configurable away, which matters for enterprise compliance contexts.

The entire workflow is YAML-configurable: agents, tools, LLMs, and routing behavior are all defined in config files, meaning teams can tune depth thresholds, swap model providers, or adjust escalation sensitivity without touching code. AI-Q is model-agnostic at the API level - it supports NVIDIA NIM™ microservices (NVIDIA Nemotron™ Super 120B, NVIDIA Nemotron™ Nano 30B), OpenAI, Anthropic, and Google, and the RAG layer plugs in NeMo Retriever for multimodal document ingestion across structured, semi-structured, and unstructured enterprise content.

What ties this back to the full stack is worth noting explicitly. AI-Q's orchestration runs on LangChain DeepAgents from Layer 4. Its sandboxed tool execution relies on OpenShell from Layer 4. Its inference runs through NVIDIA NIM on Dynamo from Layer 1. The blueprint is not a separate product sitting above the stack - it is the stack assembled into a deployable, benchmarked, enterprise-grade agent.

Layer 6 - Security & Trust: NVIDIA Nemo Guardrails – Garak

No production AI system should be deployed without adversarial testing. Garak, NVIDIA's open-source LLM red-teaming framework, addresses the security testing gap that exists throughout the industry.

Most model evaluation focuses on capability benchmarks: can the model answer factual questions, write correct code, pass professional exams? Garak asks a different set of questions: can the model be made to reveal sensitive information? Can it be manipulated into ignoring its system prompt? Does it hallucinate dangerously in specific domains? Will it comply with requests it should refuse?

The framework is organized around four conceptual layers:

Generators: adapters that connect Garak to any LLM API - OpenAI, Anthropic, Hugging Face, or local models via Ollama.
Probes: attack modules that implement specific red-teaming strategies. Garak ships with over 120 probe categories covering prompt injection, jailbreaking, data extraction, misinformation generation, and denial of service patterns.
Detectors: evaluators that assess model outputs for signs of successful attack - did the model comply with the jailbreak, did it leak training data, did it produce harmful content?
Buffs: transformations applied to probes before they reach the model, simulating the encoding, rephrasing, and obfuscation techniques that real attackers use to evade safety measures.

For agentic systems specifically, Garak's probe library includes tests for multi-turn manipulation (where an attacker gradually shifts an agent's behavior across many conversation turns) and tool-mediated attacks (where the exploit arrives through a tool result rather than a direct user message). These attack vectors are largely absent from standard safety evaluations but are exactly what production agents face.

Teams should treat Garak scans as a pre deployment gate in CI/CD, scanning models and prompt configurations before deployment, so vulnerabilities are caught before they reach production.

The Stack Is Now Visible

What GTC 2026 made clear is that building reliable agentic AI is no longer a research problem. It is an engineering problem with an increasingly well-defined solution space:

Infrastructure that handles inference at scale, disaggregating the compute so agents can run fast and cheaply across multi-node GPU clusters (Dynamo)
Sandboxed execution that enforces policy-based boundaries around every tool call, so that when an agent makes a mistake - and it will - the blast radius stays contained (OpenShell)
A context engineering discipline that determines what agents actually think with at each step - because many agent failures are context failures, not model failures
Planning architectures that give agents the ability to decompose, delegate, and persist across multi-hour, multi-step work without losing track of where they are (LangChain Deep Agents)
A deployable, benchmarked enterprise agent that assembles the full stack into something knowledge workers can actually query: routing intent, managing research depth, verifying citations, and waiting for human approval before it commits to a ten-minute investigation (NVIDIA AI-Q Blueprint)
Adversarial testing that probes every layer of the stack for the failure modes that capability benchmarks will never find (Garak)

None of these pieces is optional, and none of them stands alone. Dynamo without harness engineering gives you fast inference on the wrong inputs. Harness engineering without sandboxed execution gives you curated context feeding an agent that can still do real damage with a bad tool call. Long-horizon planning without citation verification and a human-in-the-loop gate gives you confident, well-structured answers that may be wrong. And none of it matters if the stack has never been red-teamed.

What we took away from these sessions, watching or reading about them, is that the conversation has shifted. A year ago, the question was whether agentic AI could work at all. The question now is whether your team is building it on the right foundations. For the first time, those foundations are all available, open, documented, and benchmarked. The engineering problem is hard - but it is now a known problem, which is a very different thing.

For further details contact cloud@amdocs.com

References & Further Reading