Personally, I didn’t favor the “just keep making it bigger” era of LLMs, even though I’ve been fascinated by generative AI applications and their potential outcomes from the very start in my career. From the technical aspect, it seemed like a lazy way of trying to work things out, like a brute force. I think that if we design in smarter ways and measure the outcomes properly, there are always better ways to implement and optimize any kind of solution that uses less resources more effectively. But our current world of ours is always eager to push the limits, even if it’s making not so meaningful discoveries and changes to the world we live in.
With that being said, I’m also aware that if we haven’t experimented with scaling the models, we wouldn’t have discovered how capable they can be. That era just made me feel that our exploration-exploitation dynamic was a bit off. It definitely didn’t help that the “conductors” of the AI hype train were making false claims about reaching AGI in no time if we kept scaling.
Right now we reached a point where models are excellent at passive generation but fail at most active task execution cases. An LLM can tell you exactly how to book a flight, but without an agentic architecture, it lacks the ability to navigate the API, compare real-time prices, and finalize the transaction without hallucinating a confirmation code.
As the year 2025 has come to a close, AI ecosystem started to embrace more efficient, reliable, robust, cost-effective and debuggable systems each passing day with utilizing techniques like agentic RAG for context retrieval&enrichment, MCP standard for tool-calling, test-time compute (thinking mode), human-in-the-loop validation approaches for reliable decision making and many more. Witnessing these recent tides of changes in the field resparked my interest with current generative AI applications in real world scenarios where it can make a real difference with real outcomes. So I wanted to take a step back from the details and look at the general trend that the industry is currently following. I will also make some speculations about where it will be going in another article, because why not.
“Likely” Doesn’t Mean “True”
As of today, even the most advanced generative AI systems majorly rely on large language models. And the way LLMs work during inference fundamentally is a simple text completion function that predicts the next words (or tokens) probability of occurrence of a given incomplete phrase. It’s just the next word’s likelihood is calculated within a highly complex deep neural network where it can recognize patterns, associate different parts of the given incomplete sentence with intricately engineered attention mechanisms under the hood.
These giant text generators can potentially generate results that we consider harmful or unwanted since it’s being trained on publicly available wide-n-wild world of web that we call internet. Techniques like fine-tuning and reinforcement learning with human feedback are used for driving the model outputs to match our expectations. But still, since it only relies on “likelihoods”, it may yield results that sound coherent but factually wrong. The system makes its best effort to sound “right”, not necessarily be “right”. This is where we call the term “hallucinating” and it is still a big problem in the field.
In order to mitigate this problem, the ‘context’ (the incomplete sentence) that the model is working on during inference must be enriched properly. Especially in the past 2-3 years techniques like retrieval augmented generation (RAG) and tool calling have come into play. They let the agent enrich the context and give agents abilities, “hands” to retrieve, to operate on and interact with external systems (e.g. start pipelines, send emails etc.) This is crucial for systems to reach accurate information during generation and for agentic AI applications where we expect the system to take certain actions.
Moving from Hallucination to Fact
In RAG, instead of trying to remember every single detail it has learned during training, the agent dynamically retrieves a relevant piece of information and depends its final answer off of it. Within most of the RAG applications, the knowledge base that the agent will be working with first must be transformed (typically vectorized) and kept in a database, this is like building a library for the agent. Whenever a query comes up, the agent retrieves the piece of information that is most similar to the given query semantically (usually vectoral similarity), injects it into the context and proceeds to the final generation.
However, without an agentic decision process like tool-calling, plain RAG might not be useful for every use case. Agentic RAG usages where an agent can decide if it needs a piece of information or not would be more useful for both retrieval based and general tasks that are given to the agent. For this reason, the RAG technique has lots of different variations.
Similar to the LLM’s word guessing game, RAG technique is also a probabilistic type of retrieval in many cases. Whenever a piece of information is needed about something, it retrieves the information pieces that resemble the given one and it may not match the exact needed information every single time. In agentic AI, this can potentially lead to cases where agents misinform the end-user or fail on the task directly, if implemented without proper human-in-the-loop verification processes or self-checking safeguards. For cases like these, I believe deterministic retrieval technique must be implemented within the system as well.
Connecting to the Outside World
This need for deterministic behavior brings us to the operational side of agentic AI: tool use and standardization. If RAG is the agent’s “memory” then tools are its “hands”. Until recently in order to make these models interact with external systems, every developer had to write custom glue code to connect their model to a database, a slack channel, or a file system. It was the integration equivalent of “spaghetti code”. This is where the Model Context Protocol (MCP) has become a game changer for the ecosystem in 2025.
Developers of MCP describe it like this on the very first introduction page:
“Think of MCP like a USB-C port for AI applications. Just as USB-C provides a standardized way to connect electronic devices, MCP provides a standardized way to connect AI applications to external systems.”
Before USB standards, you needed a different cable for every device; a printer, a camera, a mouse. Similarly, before MCP, connecting an LLM to a repository required a specific driver; connecting it to a SQL database required another. MCP standardizes this interface. It decouples the model from the data source.
Now, a developer can build an “MCP Server” for their internal API once, and any MCP-compliant agent can plug into it and start using those tools immediately. And practically, developing this MCP layer is not so different than implementing a well documented API. This standardization is what finally allows us to move from isolated “chatbots” to interconnected ecosystems of agents that can actually do work across different environments without custom integration for every single step.
I would also argue that MCP will supersede Agentic RAG practices in the future. Since MCP already elevates tool calling abilities, the RAG operations can be implemented within a simple MCP function. For example within an MCP layer, a deterministic search query can be executed on a structured dataset directly while working on tasks that have zero tolerance of error. And if we have an unstructured knowledge base, RAG is needed. My guess is these unstructured/structured and probabilistic/deterministic retrieval methods will be unified under a single roof in the near future.
Having access to memory (RAG) and hands (MCP) is useless if the brain controlling them acts impulsively. This brings us to the most critical shift in post-training optimization: test-time compute, often referred to as “thinking models” or “System 2” reasoning.
Planning Before Acting
In the “keep scaling” era I did mention earlier, we tried to make models smarter by training them on more data for longer periods (Training-time compute). But we realized that for complex, multi-step tasks the model doesn’t need to know more; it needs more time to think before it speaks. Test-time compute allows the model to generate a “chain of thought” where it plans, critiques, and refines its approach before outputting a single word to the user.
This approach was already being implemented within custom agentic flows on AI applications (langchain framework was popular for this), but it is also lately getting integrated within the LLMs natively, meaning we started to fine tune them to prioritize behaving that way.
In an agentic workflow, this is the difference between an employee who immediately shouts the first answer that comes to mind versus one who says, “Give me a minute,” checks the documentation, verifies the facts, and then delivers a polished answer. When we combine this “thinking” capability with tool use, we get agents that can self correct itself: they might try to retrieve data via RAG, realize the retrieval was poor, and decide to run a clearer search query; all without human intervention.
Conclusion
This combination is the foundation of the modern Agentic AI stack. By giving models memory (RAG), hands (MCP), and the time to think (Test-time compute), we aren’t just making them bigger, we are making them functional. It moves us away from the probabilistic “guessing game” of early LLMs to the reliable, robust systems that are on the rise today. The hype train might have slowed down, but the real journey of transformation is just beginning.
And in order to steer this transformational power that the world has never seen before in the right direction, for the good, we must thoroughly understand and assess each step we are considering to take.
Until next time, much love.