The Limits of LLM Based Systems

Understanding the limitatins of LLM-based AI agents, and thinking of them as sophisticated token-sequence generators rather than autonomous agents has important implications for what we should expect from these systems in the near future, as well as how and where we can best deploy them in society.

For well-defined tasks with sufficient context, these agents may indeed operate nearly autonomously, giving us an the impression they are genuine autonomous agennts.

But when confronting complex or novel problems, these systems will be expected to fail producing the desired results. Yet, they can still be valuable tools as a new type of high-bandwidth computer interface, that helps a user to turns their thoughts and intentions into text and code way faster than manual typing onle would have allowed them to.

Thinking of them as "AI assistant" might be more accurate than the "AI agents". When working on complex ideas, the usable output generated by an LLM is often the fast and partial completion of what its user intended to do. A developer who clearly understands the software architecture for a new type of application, can use an LLM assistant to dramatically accelerate implementation. But the human expert must remain in the loop to prevent the assistant from drifting toward generic, overused frameworks or producing incoherent output when working on a novel task.

This suggests (as far as LLMs and systems build on top of them are considered) a future where AI assistents augment rather than replace human expertise. In software development, data analysis, writing, research, and other knowledge work, LLM asistents already serve as powerful productivity accelerators for skilled professionals. They do not (nor migth they in the near future) eliminate the need for domain understanding required to guide them toward useful output on complex tasks.

The next decade will likely not be defined by a single "God-like" LLM-based AI model, but by how effectively we can combine these imperfect agents into reliable systems. If we can overcome the error accumulation problem through tool use and collective intelligence, we may indeed be on the path to AGI. If not, they may still be the most powerful computer interface huanity has build as yet. Allowing us to build powerful systems that can autonomously perform well deifned tasks

Even if the current technology will not further evaluate beyond 2025 it will still take years for many startups to work out the agentic scaffoldings for many processes, ad work out which processes are most monetizable. Meaning AI adoption will still take years, and the impact of it will be huge