From LLMs to Agents

Direct Implications in Enterprise IT

January 1, 2026

One of the more direct consequential implication of Agentic AI systems for large enterprises isn't technical, but organisational. And it's been hiding in plain sight, dressed up in the "way of working" process diagrams of their IT departments.

The underlying observation is straightforward. When a user comes with a question or a goal to an AI Agent, the first thing the agent has to do is find the right tool to use β€” one where the available capabilities actually fit what the user is trying to achieve. At its simplest, an agent is a router: it matches a need to a tool, decides which tools to call, in what order, with what arguments, and how to combine the results. That's it. Everything else β€” the model, system prompts, guardrails β€” are in service of getting those decisions right.

In any large enterprise IT department, a bulk of the (human-led) work consists of this. A business stakeholder shows up with a use case. Someone β€” a business analyst, a product owner, a solutions architect, a co-creation team, an integration squad β€” figures out which of the existing IT services can serve that use case. They negotiate small customisations. They wire the pieces together. They forward the user to the right managed IT service, with the right adjustments. The pattern an AI Agent follows, is structurally identical to what teams thoughout enterprise organisation have been doing for decades.

Taken seriously, the implication of this is not that organisations should add an Agentic Team next to the existing teams. An Agentic Team should ideally absorb these functions that are currently spread thinly across the organisation, and the rest of the organisation should reshape around that.

What Enterprise IT looks like today

Providing IT solutions in an Enterprise organisation starts with business teams that are in need of specific IT solutions. Then there are integration or co-creation teams that sit between these business teams and product teams, helping them to translate vague business needs into pipelines for applications or models.

The are product teams develop, own, and maintain standardised those applications, models, platforms and other IT services. The implicit contract is: product teams provide standardised services; integration teams work out the required customisation for specific use cases; business users get something that looks tailored while the underlying IT solution is shared. The goal of this separation is to be able to scale an IT department throughout the organisation, and have it cover as many different use cases as possible with a limited number of standardised servics.

This works, sort of, for a while. Then it often hits a limit, because the customisation demands keep expanding. Every new use case turns out to need some small twist on an existing service. The integration teams accumulate custom dashboards and connectors. Service teams get pulled into every customisation, because only they understand the edge cases. The boundary between "standardised platform" and "custom solution" blurs into a fog. And scaling IT solutions to cover more and use cases hits its limits without more and more people to take care of all the different customisations.

The deep problem here isn't laziness or bad process. It's fundamental limitations on the scalability of this model; service cannot be standardised well if every consumer needs it slightly differently and there's no clean way for those differences to be expressed. Solving these customisations is slow, expensive, lossy, and doesn't scale well. Every customisation is a one-off-a-kind; in the worst case nothing learns from anything else.

What Agentic AI changes

Agentic AI systems moves "the place where customisation happens", out of the product layer and into a new IT layer above it. When given one standardised tool or service, an agentic system is build to handle small custimisations on the input, vary parameters, retry, and tailor the use of the tool to a specific use case, without having to ask the tool to change. The tool stays standardised. The customisation lives in the agent.

Because fo this, Agentic AI systems structurally empower what the integration and co-creation teams have been trying to do by hand all along. These teams have been the routing layer. Connecting business teams to the right services and tools.

So the relevant question for an enterprise IT department in 2026 is not "should we add AI Agents as an extra product". The question is: given that there is now a technology layer that can do the routing that has historically been done by hand, how should the organisation restructure to put this to optimal use.

What Enterprise IT looks like with Agentic AI

The answer turns out to be reasonably clean. Once customisation can be seperated from the product layer, and combined with a new Agentic layer in the organisation that focusses on building Agentic systems, the work that used to be smeared across product teams and integration squads, sorts itself into two more clearly structured categories: building products, and integrating them into (agentic) solutions. The original overlap and mixing between these layers with different incentives, and a different definition of success, was precisely what produced the fog and limits on scalability described earlier.

The Product layer is where product teams live. Their job is to produce services, that an agentic can call as tools: well-defined, standardised, well-documented building blocks. These teams will no longer be expected to also deliver custom dashboards and bespoke pipelines to support specific business teams with their products. They are expected to deliver a wide but sharply defined catalogue of building block capabilities that compose well with each other. The interesting consequence is thse product teams get narrower and deeper. They stop chasing every use case downstream of them, and can fully focuss on the quality of their tools. The relevant KPI becomes something like "how often is our tool called successfully by an agent without human intervention?" rather than "how many projects did we deliver?"

The Agentic layer is where the customisation moves. Agentic teams own the custom development of agents β€” that take user queries and decide which services to call, in what sequence, with what guardrails, to satisfy specific business goals. These teams own the things that previously lived in tickets and meetings: which tools fit which business case, how to compose them, what the user experience around the agent looks like, how to evaluate whether the agent did the right thing. This is the layer where most of the new work lives: custom prompt engineering, observability, eval design, tool selection logic, guardrail placement, and human-in-the-loop checkpoints.

What this means for individual contributors

For engineers, analysts, and PMs in enterprise IT departments right now, the relevant question is which layer their work belongs in. The honest answer for most people is that their current job might be a mixture.

Those whose value comes from understanding how business problems map to technical capabilities belong in the agentic layer. Where their domain knowledge about which services fit which business need becomes the core asset, but expressed differently, as agent logic and tool selection. The skill that matters most is system design: which tools, in which sequence, with which guardrails, evaluated how.

Those whose value comes from understanding how a specific platform works deeply belong in the product layer, and their job is going to be designing the capabilities that agents will consume. Their deep understanding of how customisations were done in the past becomes the basis to design a broader set of tools with a more narrow scope, that require refer customisations in the future. Getting as close as possible to defining each building block product as a clean MCP-intgratable tool with sharply defined general features and observable failures.

They cannot, indefinitely, stay in the middle as a manual routing and integration function. That role is the one the agentic layer is taking.

What to expect

The underlying point is that agentic AI is not a new feature in an existing IT department. It's a technology, that enables a different way of slicing the same problem.

The routing and intrgration function that has lived largely inside humans is moving into software designed by humans. The question for an Enterprise IT department is not whether that is going to happen β€” it is happening. The organisations that thrive will be the ones that draw the new boundary deliberately rather than wait to be redrawn by it.

Set up an agentic team to build a few demos, while keeping the integration and co-creation functions doing exactly what they did before. In that world the agentic team becomes a side project, the routing function stays manual, and three years later someone writes a post-mortem about how "agentic didn't deliver." The mistake will not be in the technology. It will be in declining to actually move the boundary.

The result, when taken seriously, is a cleaner and more scalable structure than the old one. Standardised tools below, composable agents in the middle, business value coming out on top. The teams that organise themselves around this end up with sharper missions.

Continue reading:AI Allignment