The Brain of an AI Agent: Understanding Foundation Models and LLMs

  

In the previous post, we explored the seven core components that together form the anatomy of a modern AI agent. Each component has a unique responsibility, but none is more important than the brain of the agent.

 

That brain is called a Foundation Model. Today, most AI agents use a Large Language Model (LLM) as their foundation model, which is why these terms are often used interchangeably. However, they are not the same. An LLM is just one type of foundation model. As AI continues to evolve, agents are increasingly powered by multimodal foundation models that can understand text, images, audio, video, and more.

 

In this post, we'll clarify this distinction, understand how foundation models enable reasoning, explore the prompting techniques that make them behave intelligently, and discuss how architects choose the right model for different applications.

 

1. Why Every AI Agent Needs a Brain

In the previous post, we explored the seven core components that together form the anatomy of a modern AI agent. Each component has a specific responsibility, and they work together to transform a simple language model into an intelligent system capable of solving real-world problems.

 

Component	Human Analogy	Responsibility
🧠 Foundation Model	Brain	Thinks, reasons, and makes decisions
👀 Perception	Senses	Understands text, images, audio, and other inputs
🎯 Planning	Strategist	Breaks complex goals into smaller tasks
📒 Memory	Notebook	Remembers previous conversations and important information
📚 Knowledge	Library	Provides domain-specific facts and information
🛠️ Tools	Hands	Interacts with external systems, APIs, and applications

 

Although every component is important, everything begins with the Foundation Model. It is the reasoning engine that interprets user requests, understands context, decides what to do next, and coordinates all the other components. Think about what happens when you ask an AI agent: "Book me the cheapest flight to Bangalore from Chennai for next Friday."

 

The agent doesn't immediately search for flights. Instead, its brain first reasons about the request:

 

·      What is the user's goal?

·      What information is already available?

·      Is any information missing?

·      Should I ask a follow-up question?

·      Do I need to search for flights?

·      Which tool should I use?

·      How should I present the results?

 

Every one of these decisions is made by the Foundation Model before any other component is involved. Without this reasoning capability, the remaining components would have no direction. Memory wouldn't know what information to retrieve. Tools wouldn't know which API to call. Planning wouldn't know how to break down the task.

 

In other words, the Foundation Model orchestrates the entire AI agent.

 

1.1 A Human Brain Analogy

Imagine you decide to cook dinner. Your brain doesn't physically chop vegetables or stir the pan. Instead, it coordinates everything.

 

It thinks:

·      What should I cook?

·      Do I have all the ingredients?

·      Should I check the recipe?

·      What's the first step?

·      How long should I cook this?

 

Once those decisions are made, your hands perform the work, your eyes observe what's happening, and your memory helps you remember the recipe.

 

An AI agent works in a very similar way.

 

·      The Foundation Model acts as the brain.

·      Memory recalls previous conversations and facts.

·      Perception understands what the user is asking.

·      Planning organizes the work into logical steps.

·      Tools interact with external systems.

·      Action completes the task.

 

The brain doesn't perform every action itself, it decides which actions should happen.

 

1.2 A Computer CPU Analogy

Another useful way to understand the Foundation Model is by comparing an AI agent to a computer.

 

Component	Computer Analogy
Foundation Model	CPU
Memory	RAM
Knowledge	Hard Disk / SSD
Tools	Keyboard, Mouse, Network Card, APIs
Perception	Input Devices
Action	Output Devices

 

The CPU is the central processing unit of a computer. It doesn't permanently store data, that's the job of the hard disk. It doesn't temporarily hold running programs, that's the job of RAM. It doesn't connect to the internet, that's handled by the network card.

 

Instead, the CPU executes instructions and coordinates all the hardware components so they work together. The Foundation Model plays exactly the same role inside an AI agent.

 

It doesn't permanently store long-term knowledge. It doesn't directly access databases or call APIs. Instead, it reasons about the user's request and decides when to retrieve memory, consult external knowledge, use tools, or execute actions.

 

This is why many AI architects refer to the Foundation Model as the CPU of an AI agent.

 

Important: While the CPU analogy is useful, it isn't perfect. A CPU simply executes programmed instructions. A Foundation Model goes much further, it can understand language, reason about problems, generate new ideas, and make decisions based on context. The analogy helps us understand its central coordinating role, not its intelligence.

 

2. What Is a Foundation Model?

Now that we understand why every AI agent needs a brain, let's look at what that brain actually is.

 

The term Foundation Model is often confused with Large Language Model (LLM). While they are closely related, they are not exactly the same. Before we dive into LLMs, it's important to understand the broader concept of a foundation model.

 

A Foundation Model is a large artificial intelligence model that is pretrained on enormous amounts of data to learn general patterns, relationships, and knowledge about the world. Instead of being built to solve one specific problem, it serves as a general-purpose foundation that can be adapted to perform many different tasks.

 

Think of it as a student who has spent years learning mathematics, science, history, languages, and problem-solving. After gaining this broad education, the student can apply that knowledge to many different careers, such as becoming a doctor, an engineer, or a teacher.

 

Similarly, a foundation model first learns broad knowledge during training. Later, we can use that same model for many different applications without training it from scratch.

 

2.1 Why Is It Called a "Foundation" Model?

The word foundation is very intentional. Just as a building starts with a strong foundation before walls, doors, and windows are added, modern AI systems begin with a foundation model before adding specialized capabilities.

 

Instead of creating a new AI model for every application, we start with one powerful pretrained model and build on top of it.

 

For example, the same foundation model can power:

 

💬 A customer support chatbot

💻 A coding assistant

📄 A document summarizer

🌍 A language translator

✍️ A content writing assistant

🤖 An autonomous AI agent

 

The underlying model remains the same. Only the prompts, tools, memory, and surrounding application change. This is why foundation models have become the starting point for almost every modern AI application.

 

2.2 Massive Pretraining

The intelligence of a foundation model comes from a process called pretraining. During pretraining, the model is exposed to an enormous amount of data collected from diverse sources, such as:

 

·      Books

·      Articles

·      Websites

·      Scientific papers

·      Source code

·      Technical documentation

·      Conversations

·      Audio files

·      Video files

·      And many other publicly available datasets

 

This training process can involve trillions of words or tokens and requires thousands of powerful GPUs running continuously for weeks or even months. The objective is not to memorize every sentence. Instead, the model learns patterns such as:

 

·      How language is structured

·      Relationships between words and concepts

·      Grammar and syntax

·      Common reasoning patterns

·      Programming languages

·      Facts about the world

·      Different writing styles

 

Over time, the model becomes remarkably good at predicting what information is most likely to come next based on the context it has been given. This ability forms the basis of everything the model can do.

 

2.3 One Model, Many Tasks

The biggest advantage of a foundation model is its versatility. Unlike traditional machine learning models, which are trained for a single task, a foundation model can perform many different tasks using the same underlying intelligence.

 

For example, with different instructions, it can:

 

·      Answer questions

·      Summarize long documents

·      Write software code

·      Translate between languages

·      Generate emails

·      Explain complex concepts

·      Analyze documents

·      Assist with research

·      Plan projects

·      Power intelligent AI agents

 

No retraining is required. We simply provide a different prompt or combine the model with components such as memory, knowledge bases, and tools.

 

This flexibility is one of the biggest reasons foundation models have transformed the field of artificial intelligence.

 

2.4 Foundation Models Are Not Limited to Language

Another common misconception is that all foundation models are language models. In reality, Large Language Models (LLMs) are just one category of foundation models.

 

Today, foundation models exist for many different types of data, including:

 

Type	Examples	Primary Capability
Large Language Models (LLMs)	GPT, Claude, Llama, Qwen	Understand and generate text
Vision Models	DINOv2, SAM	Understand images
Image Generation Models	Stable Diffusion, FLUX	Generate images
Speech Models	Whisper	Understand and transcribe speech
Multimodal Models	GPT, Gemini	Understand and reason across text, images, audio, and more

 

As AI continues to evolve, many modern agents are moving beyond text-only interactions. They can now understand images, interpret audio, analyze videos, and combine multiple types of information to solve complex problems.

 

This is why the broader term Foundation Model is becoming increasingly important in AI architecture.

 

In summary, A Foundation Model is the general-purpose intelligence at the heart of modern AI systems. It is pretrained on massive and diverse datasets, learns broad patterns rather than a single task, and serves as the starting point for building countless AI applications.

 

Most AI agents today use a Large Language Model (LLM) as their foundation model. In the next section, we'll clarify the relationship between these two terms and answer a common question:

 

3. Foundation Models vs Large Language Models (LLMs)

One of the most common misconceptions in AI is that Foundation Models and Large Language Models (LLMs) are the same thing. Although these terms are often used interchangeably, they are not identical.

 

The easiest way to understand their relationship is this: Every Large Language Model (LLM) is a Foundation Model, but not every Foundation Model is an LLM.

 

In other words, LLMs are a subset of Foundation Models.

 

Think of it like this:

·      Every car is a vehicle.

·      But not every vehicle is a car.

 

Similarly,

·      Every LLM is a Foundation Model.

·      But Foundation Models also include models that understand images, audio, video, and other types of data.

 

3.1 Foundation Models: The Bigger Family

A Foundation Model is a general-purpose AI model that has been pretrained on massive amounts of data. It provides a strong foundation that can be adapted to perform many different tasks. Foundation models are not limited to language. Depending on the type of data they are trained on, they can specialize in different domains.

 

Some examples include:

·      Models that understand and generate text

·      Models that recognize objects in images

·      Models that generate realistic images

·      Models that understand speech

·      Models that process multiple types of data, such as text and images together

 

3.2 Large Language Models: A Type of Foundation Model

A Large Language Model (LLM) is a foundation model that has been specifically trained to understand and generate human language. Its primary strength is working with text.

 

An LLM can:

·      Answer questions

·      Generate articles

·      Write software code

·      Translate languages

·      Summarize documents

·      Explain concepts

·      Hold conversations

·      Reason through complex problems

 

Because language is the primary way humans communicate, LLMs have become the most widely used foundation models today.

 

3.3 Comparing Foundation Models and LLMs

Foundation Model	Large Language Model
A broad category of pretrained AI models.	A specific type of foundation model focused on language.
Can work with text, images, audio, video, or multiple modalities	Primarily understands and generates text (and often code).
Includes language, vision, speech, image generation, and multimodal models.	Includes models such as GPT, Claude, Llama, Qwen, and Mistral.
Designed to be adapted for many downstream AI applications.	Designed for language understanding, reasoning, and text generation.
Represents the broader architectural concept.	Represents one implementation of that concept.

 

You can think of the relationship like this:

 

Foundation Models
│
├── Large Language Models (LLMs)
│     ├── GPT
│     ├── Claude
│     ├── Llama
│     └── Qwen
│
├── Vision Models
│
├── Speech Models
│
├── Image Generation Models
│
└── Multimodal Models

 

Notice that LLMs are just one branch of the Foundation Model family.

 

3.4 Why Do AI Agents Mostly Use LLMs Today?

If foundation models include many different types of models, why do almost all AI agents use Large Language Models as their brain?

 

The answer is surprisingly simple, most human instructions are expressed in natural language.

 

Whether we ask:

 

·      Summarize this report.

·      Book a flight to Delhi.

·      Write a Python program.

·      Analyze these sales figures.

·      Plan my vacation.

·      Explain quantum computing.

 

Every request begins as language. To complete these tasks, an AI agent must first understand what the user wants. It must interpret intent, reason about the request, plan the required steps, decide whether to use tools, and finally generate a meaningful response.

 

These are exactly the strengths of an LLM. An LLM doesn't simply generate text, it acts as the reasoning engine that coordinates the entire agent. It decides:

 

·      What is the user's goal?

·      Is additional information needed?

·      Should memory be consulted?

·      Should an external tool or API be called?

·      What should happen next?

 

This ability to understand language, reason over context, and orchestrate actions makes LLMs the natural choice for today's AI agents.

 

3.5 The Future Is Moving Beyond Language

Although today's agents are primarily powered by LLMs, the future is increasingly multimodal. Modern foundation models can already understand and reason across multiple types of information, including:

 

·      Text

·      Images

·      Audio

·      Video

·      Documents

·      Screenshots

·      Charts and diagrams

 

Imagine asking an AI agent: "Look at this dashboard, listen to this meeting recording, read the attached report, and summarize the key business risks."

 

This task requires understanding multiple forms of data, not just text. As multimodal foundation models become more capable, future AI agents will rely on them rather than language-only models. This is why architects increasingly use the broader term Foundation Model when discussing AI system design.

 

In summary, A Foundation Model is the broad category of pretrained AI models that serve as the foundation for modern AI applications. A Large Language Model (LLM) is one specific type of foundation model that specializes in understanding and generating human language.

 

Today, most AI agents use LLMs because language is the primary interface between humans and machines. However, as AI systems continue to evolve, agents will increasingly be powered by multimodal foundation models capable of reasoning across text, images, audio, video, and more.

 

In the next section, we'll explore how these models learn during pretraining and why that training enables them to perform such a wide variety of tasks.

 

4. How Foundation Models Learn?

Now that we understand what a Foundation Model is, the next question is naturally: How does a Foundation Model become so intelligent?

 

Can it really read the entire internet? Does it memorize millions of books? How can it answer questions about history, write software code, explain science, and even compose poems?

 

The answer lies in a process called pretraining.

 

4.1 Learning from Massive Datasets

Before a foundation model can answer questions or assist users, it must first learn about the world. This learning happens during a phase called pretraining, where the model is exposed to an enormous collection of data from many different sources.

 

These datasets may include:

 

·      📚 Books

·      🌐 Websites

·      📰 News articles

·      📄 Research papers

·      💻 Source code

·      📖 Technical documentation

·      💬 Public conversations

·      📜 Reference materials

 

And many other publicly available datasets. Rather than studying one subject at a time, the model learns from a wide variety of topics simultaneously.

 

You can think of it as a student who spends years reading books from a gigantic library that contains information about science, history, mathematics, programming, literature, medicine, business, and countless other subjects.

 

The more diverse the learning material, the broader the model's understanding becomes.

 

4.2 Does It Memorize Everything?

A common misconception is that a foundation model simply memorizes everything it reads. That's not how it works.

 

Imagine reading hundreds of detective novels. After a while, you begin to notice patterns:

 

·      Every story introduces important characters early.

·      A mystery is usually solved near the end.

·      Clues appear before the final reveal.

·      Certain writing styles are common.

 

Even if you've never read a particular detective novel before, you can often predict what might happen next because you've learned the patterns, not memorized every story.

 

Foundation models learn in a similar way. Instead of storing every sentence exactly as it appears, they learn:

 

·      How language is structured

·      How ideas are connected

·      Which words frequently appear together

·      How people ask questions

·      How problems are solved

·      How programming languages are written

 

Their intelligence comes from recognizing these patterns across billions or even trillions of examples.

 

4.3 Learning from Source Code

Foundation models don't learn only from natural language. Many are also trained on large collections of source code written in programming languages such as Java, Python, JavaScript, C++, Go, and many others.

 

During training, the model begins to recognize coding patterns, including:

 

·      Programming syntax

·      Common algorithms

·      Design patterns

·      Error handling techniques

·      API usage

·      Documentation styles

 

This is why modern AI assistants can generate code, explain existing programs, identify bugs, and even suggest improvements. To the model, code is simply another structured language with its own rules and patterns.

 

4.4 Learning Through Pattern Recognition

The real strength of a foundation model comes from its ability to recognize patterns. Suppose the model repeatedly encounters sentences like:

 

·      Birds can fly

·      Eagles can fly

·      Sparrows can fly

·      Penguins are birds that cannot fly

 

Over time, the model learns that:

 

·      Most birds can fly

·      Penguins are an exception

 

It doesn't memorize these exact sentences. Instead, it builds an understanding of the relationships between concepts. This same process applies to programming, mathematics, history, medicine, and almost every other domain.

 

The model continuously discovers connections, similarities, and relationships across enormous amounts of information.

 

4.5 What Is Statistical Learning?

At this point, you might wonder, if the model doesn't memorize information, how does it know what to say next?

 

The answer is statistical learning. This simply means the model learns which patterns are most likely to occur together.

 

For example, consider the sentence:

 

The capital of France is...

 

After seeing millions of similar examples during training, the model has learned that the word "Paris" is overwhelmingly more likely to follow than words like "Banana" or "Elephant."

 

It isn't looking up the answer in a database. Instead, it predicts the most probable continuation based on everything it learned during training. This same principle applies to much more complex situations.

 

When you ask: "How do I optimize a Java application?"

 

the model predicts the sequence of words that is most likely to form a helpful and accurate explanation based on patterns it learned from countless programming books, documentation, tutorials, and code examples.

 

4.6 Learning by Repetition

Imagine teaching a child to complete the sentence:

 

"The sky is..."

 

At first, the child may guess incorrectly. You correct the answer: "The sky is blue." After hearing thousands of similar examples, the child begins answering correctly without being reminded.

 

Foundation models learn in a similar way, but on a much larger scale.

 

During training:

·      The model predicts the next word.

·      The prediction is compared with the actual word.

·      If the prediction is wrong, the model adjusts itself slightly.

·      This process repeats billions of times.

 

Each small correction helps the model to improve. After trillions of these learning steps, the model becomes remarkably good at understanding language, answering questions, writing code, and solving problems.

 

4.7 From Prediction to Intelligence

At first glance, predicting the next word may sound surprisingly simple. However, when this process is repeated across trillions of words from diverse sources, the model begins to capture complex relationships, facts, reasoning patterns, and language structures.

 

What starts as next-word prediction gradually develops into abilities that appear remarkably intelligent.

 

It can:

·      Explain difficult concepts

·      Write software

·      Summarize documents

·      Translate languages

·      Solve logical problems

·      Hold natural conversations

 

This is why foundation models often seem to understand the world, even though their core learning process is based on recognizing patterns and predicting what comes next.

 

We'll explore this fascinating idea in the next section, where we'll see how simple next-token prediction gives rise to reasoning, problem-solving, and many of the intelligent behaviours we observe in modern AI systems.

 

5. The Secret Behind Their Intelligence: Next-Token Prediction

Now we've reached one of the most fascinating concepts in artificial intelligence. After learning that foundation models are trained on enormous amounts of data, many people ask: How can a model write essays, generate software code, answer questions, and even solve complex problems?

 

The answer may surprise you. At its core, a Large Language Model performs one simple task, it predicts what comes next. More specifically, it predicts the next token. It sounds almost too simple to be true, but this single capability is the foundation of everything an LLM can do.

 

5.1 What Is a Token?

Before we understand next-token prediction, we first need to know what a token is. A token is the smallest unit of text that a language model processes.

 

Depending on the text, a token might represent:

 

·      A complete word

·      Part of a word

·      A punctuation mark

·      A number

·      A symbol

 

For example, consider the sentence, "Artificial Intelligence is transforming the world".

 

A language model doesn't necessarily see this as one complete sentence. Instead, it breaks it into smaller pieces called tokens.

 

Artificial | Intelligence | is | transforming | the | world | .

 

Or, for longer or uncommon words, it might split them into even smaller parts. The exact tokenization varies from model to model, but the important idea is this: LLMs think in tokens, not sentences.

 

5.2 Predicting One Token at a Time

Now imagine the model receives the following input: The capital of France is.

 

At this point, the model asks itself, "What token is most likely to come next?". Based on everything it learned during training, the highest probability answer is: Paris

 

Once it predicts Paris, it doesn't stop.

 

Now the sentence becomes: The capital of France is Paris

 

Again it asks: "What comes next?" for the sentence "The capital of France is Paris"

 

This process continues until the complete response has been generated. The model builds an entire paragraph one token at a time.

 

5.3 Why Repeating This Billions of Times Makes the Model Smart?

At first, predicting one token seems far too simple to explain the impressive capabilities of modern AI. The secret lies in scale. During pretraining, the model performs this prediction task billions or even trillions of times.

 

For every sentence in the training data, it repeatedly asks:

 

·      What word comes next?

·      Was my prediction correct?

·      If not, how should I improve?

 

Each prediction slightly adjusts the model. Each correction helps it recognize better patterns. After repeating this process trillions of times across books, websites, research papers, and source code, the model gradually learns:

 

·      Grammar

·      Vocabulary

·      Facts

·      Programming syntax

·      Logical relationships

·      Writing styles

·      Common reasoning patterns

 

In other words, intelligence emerges from an enormous number of tiny learning steps.

 

5.4 Why This Creates Fluent Language

Think about how children learn to speak. No one teaches them every possible sentence. Instead, they hear millions of conversations over many years. Gradually, they learn:

 

·      Which words belong together,

·      how sentences are structured,

·      and how ideas naturally flow.

 

Foundation models learn in a surprisingly similar way. Because they have seen billions of examples during training, they become extremely good at predicting what sounds natural. That's why they can produce responses that are:

 

·      grammatically correct,

·      easy to read,

·      coherent,

·      and contextually relevant.

 

They aren't selecting sentences from a database. They are generating new text one token at a time based on learned language patterns.

 

5.5 Why Code Generation Works

The same idea applies to programming. To a language model, source code is simply another language with its own grammar and patterns.

 

Consider the following prompt:

public int add(int a, int b) {

   

Based on millions of Java programs seen during training, the model predicts:

 

return a + b;

   

followed by:

}

 

It isn't searching for this function in a database. Instead, it has learned the statistical patterns of Java programming. This is why modern LLMs can:

 

·      generate code,

·      explain code,

·      detect bugs,

·      complete partially written programs,

·      and even suggest better implementations.

 

5.6 Why Conversations Feel So Natural ?

When you're chatting with an AI assistant, it often feels as though you're talking to another person. That's because the model continuously predicts the next most appropriate token based on:

 

your current message,

·      the previous conversation,

·      the system instructions,

·      and everything it has learned during training.

 

Every response is generated dynamically. If you ask the same question twice, you may receive slightly different wording because the model isn't retrieving a fixed answer, it is generating a new response each time.

 

This makes conversations feel flexible, engaging, and surprisingly human.

 

5.7 It's More Than Just Guessing

At this point, you might think: "So the model is just guessing the next word?"

 

Technically yes, but it is an extremely well-informed prediction. That prediction is based on patterns learned from trillions of tokens across countless domains.

 

It considers the entire context of the conversation, the relationships between concepts, grammar, facts, programming knowledge, and much more before deciding what token should come next.

 

What appears to us as reasoning, explanation, creativity, or problem solving is ultimately built on this sophisticated prediction process.

 

In summary, everything a Large Language Model does from writing stories and generating code to answering questions and holding conversations starts with a surprisingly simple idea: Predict the next token.

 

By repeating this prediction task trillions of times during training, the model learns the patterns of language, code, and knowledge. When you interact with it, it generates responses one token at a time, producing fluent language, useful code, and natural conversations.

 

In the next section, we'll explore an even more interesting question: If an LLM is only predicting the next token, why does it appear to reason, plan, and solve problems like an intelligent system?

 

6. Why Next-Token Prediction Looks Like Reasoning?

At this point, you might have an important question. If a Large Language Model is only predicting the next token, why does it seem so intelligent?

 

How can something that simply predicts the next word:

 

·      Solve mathematics problems?

·      Write software?

·      Explain scientific concepts?

·      Plan a vacation?

·      Analyze legal documents?

·      Create business strategies?

 

It almost feels like the model is thinking. This is one of the most fascinating discoveries in modern artificial intelligence. The answer is that reasoning is not explicitly programmed into the model. Instead, it emerges from the combination of large-scale training, a powerful architecture, and vast amounts of data.

 

Let's understand what that means.

 

6.1 Intelligence Emerges from Scale

Imagine teaching a child only ten English sentences. The child might learn those sentences by heart, but they wouldn't be able to hold a meaningful conversation. Now imagine the same child spending years reading books, listening to conversations, attending school, and interacting with people.

 

Over time, something remarkable happens. The child doesn't just memorize information—they begin to understand language, recognize patterns, make connections, and solve new problems they've never seen before.

 

Foundation models behave in a similar way. During training, they don't see just thousands or millions of examples. They process billions or even trillions of tokens from books, websites, articles, research papers, and source code. As the scale of data and training increases, entirely new capabilities begin to appear. These capabilities were not explicitly programmed by engineers. They simply emerge as a result of learning from an enormous amount of diverse information.

 

This phenomenon is known as emergent abilities.

 

What Are Emergent Abilities?

An emergent ability is a capability that was not directly taught during training but appears naturally once the model becomes large enough and has learned from enough data.

 

For example, engineers never taught an LLM:

 

·      Here is how to summarize a document.

·      Here is how to write Python code.

·      Here is how to explain quantum physics.

·      Here is how to solve Sudoku puzzles.

 

Instead, the model learned patterns from millions of similar examples. As a result, it developed the ability to perform entirely new tasks. Some common emergent abilities include:

 

·      Writing software code

·      Solving logical problems

·      Translating languages

·      Summarizing documents

·      Answering complex questions

·      Explaining difficult concepts

·      Following multi-step instructions

 

These capabilities seem to appear as models become larger and are trained on more diverse data.

 

6.2 Inference: Connecting the Dots

One of the most impressive abilities of modern foundation models is inference. Inference means drawing a conclusion from the information that is available, even when the answer is not stated directly.

 

For example, consider this prompt, "Ravi forgot his umbrella. A few minutes later, he arrived at the office completely wet. What probably happened?"

 

The passage never says, "It was raining." Yet most people immediately understand that it probably rained. An LLM can make the same connection. It combines the clues:

 

·      Forgot umbrella

·      Arrived wet

 

and infers the most likely explanation. This ability comes from recognizing patterns seen across countless similar examples during training.

 

6.3 Planning Without Being Explicitly Taught

Foundation models can also generate surprisingly good plans.

 

Suppose you ask, Plan a three-day trip to Paris with a budget of €800.

 

The model may produce something like:

 

Day 1

Visit the Eiffel Tower

Explore the Seine River

Dinner nearby

 

Day 2

Louvre Museum

Notre-Dame

Local café

 

Day 3

Montmartre

Shopping

Return journey

 

No one programmed these exact travel plans. Instead, the model has learned common planning patterns from travel blogs, itineraries, guidebooks, and user discussions. It recognizes what usually comes first, what activities fit together, and how to organize them into a logical sequence. Again, this capability emerges from pattern recognition.

 

6.4 Pattern Recognition Is the Real Superpower

At its core, a foundation model is an extraordinary pattern recognition engine. It constantly identifies relationships such as:

 

·      Questions and their answers

·      Problems and their solutions

·      Causes and their effects

·      Instructions and their outcomes

·      Code and its expected behaviour

·      Conversations and natural responses

 

For example, after seeing millions of examples, it learns patterns like:

 

·      Java classes often contain methods.

·      Recipes begin with ingredients.

·      News articles usually start with a headline.

·      Emails often begin with a greeting.

·      Programming errors are followed by debugging steps.

 

When you ask a new question, the model doesn't search for an exact answer. Instead, it recognizes which learned patterns best match your request and generates a response based on those patterns.

 

6.5 Why It Appears Intelligent?

When we observe an AI assistant:

 

·      answering questions

·      explaining complex topics

·      writing code

·      creating plans

·      and engaging in conversations

 

it's natural to think, The model is thinking just like a human. In reality, what we're observing is the result of:

 

·      enormous amounts of training data

·      billions of learned patterns

·      sophisticated neural network architecture

·      and next-token prediction performed repeatedly.

 

These ingredients together produce behaviour that looks remarkably similar to reasoning. This doesn't mean the model thinks exactly like humans. Rather, it has become exceptionally good at predicting responses that are logical, coherent, and contextually appropriate.

 

To us, that often feels indistinguishable from reasoning.

 

6.6 Where Does This Reasoning Come From?

It's important to understand that reasoning isn't the result of a single breakthrough. Instead, it emerges from three key ingredients working together:

 

·      Scale: The model is trained on enormous amounts of diverse data containing knowledge from many different domains.

·      Architecture: Modern transformer architectures enable the model to understand long-range relationships, context, and complex language structures.

·      Training: By repeatedly predicting the next token and learning from its mistakes billions of times, the model gradually develops increasingly sophisticated capabilities.

 

Individually, each ingredient is powerful. Together, they give rise to behaviours that resemble human reasoning.

 

Although foundation models often appear intelligent, it's helpful to think of them as powerful reasoning engines, not human minds. They excel at recognizing patterns, generating logical responses, and making useful inferences based on what they have learned.

 

However, they do not possess consciousness, emotions, beliefs, or personal experiences. Their "reasoning" is the result of learned statistical relationships and sophisticated computation not human thought.

 

Understanding this distinction helps us appreciate both the remarkable capabilities and the current limitations of modern AI systems.

 

In summary, A Large Language Model doesn't contain a built-in reasoning engine that was explicitly programmed by humans. Instead, reasoning emerges from learning patterns across trillions of tokens, using a powerful transformer architecture, and improving through billions of prediction-and-correction cycles.

 

This combination gives rise to emergent abilities such as inference, planning, code generation, and problem-solving. While the model is fundamentally predicting the next token, the scale and sophistication of that prediction process make its responses appear remarkably intelligent.

 

In the next section, we'll see how AI engineers take these raw capabilities and shape them into reliable, goal-oriented behavior through prompt engineering.

 

7. Programming the Brain with Prompts

So far, we've learned that a Foundation Model is pretrained on enormous amounts of data and has developed impressive capabilities such as language understanding, reasoning, planning, and code generation.

 

However, this raises an important question. If a Foundation Model knows so much, how do we make it perform the specific task we want?

 

For example, the same model can:

 

·      Answer a student's homework questions

·      Write software code

·      Act as a customer support assistant

·      Help doctors summarize medical reports

·      Assist lawyers with legal research

·      Plan a vacation itinerary

 

How can one model perform so many completely different jobs? The answer is prompt engineering.

7.1 What Is Prompt Engineering?

Prompt engineering is the process of giving clear instructions to a Foundation Model so that it behaves in the desired way. Think of a Foundation Model as a highly educated person who has read millions of books.

 

That person possesses enormous knowledge, but before they begin working, you still need to explain:

 

·      What their role is

·      What task they should perform

·      What rules they should follow

·      What the expected output should look like

 

Without clear instructions, even a very knowledgeable person may misunderstand your expectations. The same is true for AI. A Foundation Model has broad knowledge, but prompts guide how that knowledge should be used.

 

7.2 From Trained Intelligence to Controlled Intelligence

Training gives a model general intelligence. Prompts transform that intelligence into task-specific behaviour.

 

Think of it this way. Training teaches the model what it knows. Prompting tells the model how to use what it knows.

 

For example, imagine hiring a highly experienced software engineer. On their first day, you don't simply say: "Start working."

 

Instead, you explain:

 

·      Your company's coding standards.

·      The project requirements.

·      The technology stack.

·      The deadlines.

·      The expected quality.

 

Only then can the engineer perform effectively. Foundation Models work in much the same way. The training process builds the intelligence. The prompt provides the direction.

 

7.3 The Different Types of Prompts

Modern AI applications usually provide the model with more than one prompt. Each prompt has a different responsibility. Let's look at the most common ones.

 

a. System Prompt

The System Prompt defines the overall behavior of the AI. It tells the model:

 

·      Who it is

·      How it should behave

·      What tone to use

·      What responsibilities it has

·      What it should avoid

 

You can think of it as the model's permanent job description.

 

For example, You are an experienced Java Architect. Explain concepts using simple English. Provide production ready examples and avoid unnecessary jargon.

 

Every response generated by the model will follow these instructions unless overridden by higher priority rules.

 

In AI agents, the system prompt often defines:

 

·      The agent's personality

·      Available tools

·      Safety rules

·      Decision-making style

·      Reasoning approach

 

It serves as the foundation for all future interactions.

 

b. Developer Prompt

The Developer Prompt contains instructions provided by the application developer. Unlike the system prompt, which defines the agent's overall identity, the developer prompt specifies how the application should behave.

 

For example:

·      Always answer in Markdown

·      Never reveal internal implementation details

·      If information is missing, ask a follow-up question

·      Format code using Java

·      Cite reliable sources when available

 

These instructions ensure the model behaves consistently across all users of the application.

 

In many AI applications, users never see these prompts, but they play a critical role in shaping the model's responses.

 

c. User Prompt

The User Prompt is the request entered by the end user.

 

Examples include:

·      Explain Kubernetes

·      Write a Spring Boot application

·      Summarize this PDF

·      Plan a three-day vacation in Japan

 

Unlike the system and developer prompts, which remain relatively stable, the user prompt changes with every interaction. This is the part of the conversation that expresses the user's immediate goal.

 

At the heart of every prompt are instructions. Instructions clearly describe the task the model should perform.

 

Instead of asking "Explain Java",  You could provide more specific instructions like "Explain Java to a beginner using simple English and include one real-world example".

 

The more precise your instructions, the more useful the response is likely to be. Good instructions reduce ambiguity and help the model produce outputs that better match your expectations.

 

7.4 Constraints: Defining the Rules

In addition to instructions, prompts often include constraints. Constraints define what the model must or must not do.

 

Examples include:

·      Respond in fewer than 200 words

·      Use bullet points

·      Do not include personal opinions

·      Explain using simple English

·      Generate only valid SQL

·      Do not access external information

 

Constraints improve consistency, reliability, and safety. Without constraints, the same prompt might produce responses with different lengths, styles, or formats.

 

7.5 Goals: Defining the Desired Outcome

Finally, every prompt should have a clear goal. The goal explains what success looks like.

 

For example, instead of saying "Summarize this report", You might say "Summarize this report for senior executives in five bullet points highlighting business risks".

 

Now the model understands not only what to do but also why it is doing it and who the output is intended for. Clear goals help the model prioritize the most relevant information.

 

7.6 Putting It All Together

Imagine building an AI travel assistant. The model might receive prompts like these:

 

System Prompt: You are a professional travel advisor who provides practical and budget-friendly recommendations.

 

Developer Prompt: Always present itineraries in chronological order. Ask for missing travel dates before making recommendations.

 

User Prompt: Plan a five-day trip to Italy with a budget of €1,200.

 

Together, these prompts define:

 

·      Who the AI is

·      How it should behave

·      What the user wants

·      How the final response should be structured

 

This layered prompting approach is what allows a single Foundation Model to power thousands of completely different AI applications.

 

Prompt Engineering Is Programming Without Traditional Code

One of the biggest shifts in modern AI development is that developers increasingly program behavior through prompts instead of writing traditional algorithms. Rather than implementing every decision in code, developers describe the desired behavior using natural language.

 

For many AI powered applications, prompts become an essential part of the application's logic. This is why prompt engineering is often described as programming the brain of an AI system.

 

8. Reasoning Patterns Used by AI Agents

In the previous section, we learned that prompts are much more than simple questions. They define the role of the AI, provide instructions, set constraints, and establish goals.

 

But prompts can do something even more powerful. They can influence how the model approaches a problem. In other words, prompts don't just tell the model what to do—they can also guide how to think.

 

This is especially important for AI agents, which often need to solve complex problems, make decisions, and perform multiple steps before arriving at a final answer.

 

Over the past few years, researchers have developed several prompting techniques that significantly improve the reasoning abilities of Large Language Models.

 

Two of the most widely used reasoning patterns are:

 

·      Chain of Thought (CoT)

·      ReAct (Reason + Act)

 

Let's understand each of them.

 

8.1 Chain of Thought (CoT)

Imagine your mathematics teacher asking you to solve a difficult problem. Instead of writing only the final answer, the teacher says: "Show your work."

 

By writing every step, it's much easier to:

 

·      identify mistakes

·      verify calculations

·      and arrive at the correct answer.

 

The same idea applies to Large Language Models. Instead of asking the model to immediately produce an answer, we encourage it to reason through the problem step by step.

 

This approach is called Chain of Thought (CoT).

 

Why Thinking Step by Step Helps?

Complex problems are rarely solved in a single leap. Humans naturally break them into smaller pieces. For example, when planning a vacation, you might think:

 

·      Where do I want to go?

·      What's my budget?

·      How many days do I have?

·      Which flights are available?

·      Which hotel fits my budget?

 

Only after answering these smaller questions you make your final decision. Chain of Thought encourages the language model to follow a similar process. Rather than jumping directly to an answer, it generates intermediate reasoning before producing the final response.

Umbrella Example

Suppose we ask the model, "Should I take an umbrella today in London?"

Without any reasoning guidance, the response might simply be: Yes.

 

While technically correct, this answer isn't very helpful because it doesn't explain why. Now imagine we modify the prompt slightly:

 

Think step by step before answering.

 

The model may reason like this:

 

·      The user wants advice about carrying an umbrella

·      To answer this, I need to consider today's weather

·      The weather forecast indicates light rain

·      People usually carry umbrellas when rain is expected

·      Therefore, taking an umbrella is recommended

 

Final answer: Yes, you should take an umbrella because light rain is expected today.

 

Notice what changed. The underlying model is exactly the same. The only difference is the prompt encouraged it to think through the problem before responding.

 

This often leads to more accurate and more reliable answers, especially for tasks involving logic, mathematics, planning, or multi-step reasoning.

 

Intermediate Reasoning Leads to Better Decisions

Chain of Thought provides two major advantages.

 

·      First, it encourages the model to break complex problems into manageable steps.

·      Second, it makes the reasoning process more structured and less likely to overlook important details.

 

This is particularly valuable for tasks such as:

·      Mathematical calculations

·      Logical reasoning

·      Planning projects

·      Debugging software

·      Solving business problems

·      Multi-step decision making

 

Rather than making one large prediction, the model builds its answer gradually, reducing the chance of errors. For AI agents, these intermediate reasoning steps often become the foundation for planning and task decomposition.

8.2 ReAct (Reason + Act)

Chain of Thought improves reasoning. But real AI agents often need to do something more. They don't just think. They also interact with the outside world.

 

For example, suppose you ask, What's the weather in London today, and should I carry an umbrella?

 

The model cannot accurately answer this using reasoning alone because today's weather changes every day.

 

Instead, it needs to:

·      think about what information is required,

·      retrieve that information from an external source,

·      and then reason about the result.

 

This is where ReAct comes in.

 

Think → Act → Observe → Think Again

ReAct stands for Reason + Act. Instead of producing a single answer, the agent repeatedly cycles through four stages:

 

Think

   ↓

Act

   ↓

Observe

   ↓

Think Again

 

·      Think: The user is asking whether they should carry an umbrella. To answer correctly, I first need today's weather.

·      Act: The agent uses a weather API to retrieve the latest forecast

·      Observe: The API returns 'Light rain, 18°C'

·      Think Again: The agent reasons "Since light rain is expected, carrying an umbrella is a good idea".

 

Finally, it responds: Yes, you should carry an umbrella because light rain is forecast today. Notice that the reasoning happened before and after using the external tool. This ability to combine reasoning with actions is what makes modern AI agents far more capable than a standalone chatbot.

 

Why ReAct Is So Powerful?

Many real-world tasks require information that isn't available inside the model itself.

 

For example, an AI agent may need to:

 

·      Search the web

·      Read a PDF

·      Query a database

·      Execute SQL

·      Call an API

·      Run Python code

·      Book a flight

·      Send an email

 

ReAct allows the model to decide:

 

·      What information is missing?

·      Which tool should I use?

·      What did the tool return?

·      What should I do next?

 

This cycle continues until the agent has enough information to complete the user's request. In other words, ReAct transforms a language model into an interactive problem solver.

 

Chain of Thought vs ReAct

Although both techniques improve reasoning, they solve different kinds of problems.

Chain of Thought	ReAct
Focuses on internal reasoning.	Combines reasoning with external actions.
Thinks step by step before answering.	Thinks, uses tools, observes results, and continues reasoning.
Best for logical, mathematical, and planning tasks	Best for dynamic tasks that require real-time information or tool usage.
Does not interact with external systems.	Frequently calls APIs, databases, search engines, and other tools.

 

You can think of it this way:

·      Chain of Thought teaches the model how to think better.

·      ReAct teaches the model how to think and interact with the world.

 

Modern AI agents use many advanced reasoning techniques, including planning, reflection, self-correction, tool orchestration, and multi-agent collaboration.

 

In this introductory article, our goal is simply to understand that prompting is about much more than asking questions, it can shape the way an AI agent reasons and solves problems.

 

9. Choosing the Right Foundation Model: An Architect's Decision

So far, we have explored how Foundation Models work, how they learn, how they generate responses, and how prompts can shape their behavior.

 

Now let's move from understanding AI models to designing AI systems. As an AI architect, one of the most important decisions you will make is: Which Foundation Model should power my AI application?

 

At first glance, this may seem like a simple question, just choose the most intelligent model available. But in real-world systems, architecture decisions are rarely that simple. The most powerful model is not always the best choice.

 

A model that can solve extremely complex reasoning problems may also:

 

·      Cost significantly more per request

·      Respond slower

·      Require more computing resources

·      Create challenges around data privacy

 

Similarly, the cheapest and fastest model may not have enough intelligence for complex business problems. The architect's job is not to find the smartest model. The architect's job is to find the right balance between capability, cost, speed, and privacy based on the application's needs.

 

9.1 The Four Architectural Trade-offs

Choosing a Foundation Model is a continuous balancing act between four important dimensions. These trade-offs define the behavior, cost, and operational characteristics of your AI system.

 

a. Capability: How Smart Is the Model?

Capability represents the intelligence level of the model. It answers questions like:

 

·      Can it understand complex instructions?

·      Can it solve multi-step problems?

·      Can it reason about ambiguous situations?

·      Can it generate high-quality code?

·      Can it handle complex planning tasks?

 

A highly capable model is better at tasks requiring deeper reasoning.

 

Examples:

·      Software architecture design

·      Scientific research

·      Complex data analysis

·      Advanced coding assistance

·      Business strategy generation

 

However, higher capability usually comes with higher cost and latency. A simple chatbot answering frequently asked questions does not always need the most powerful reasoning model.

 

b. Cost: How Much Does Each Request Cost?

Every interaction with a Foundation Model consumes computational resources. The cost depends on factors such as:

 

·      Model size

·      Number of tokens processed

·      Number of requests

·      Response length

·      Hosting approach

 

This becomes especially important in AI agents. A traditional chatbot may call an LLM once. But an AI agent may:

 

·      Analyze the request

·      Retrieve information

·      Call multiple tools

·      Generate intermediate reasoning

·      Verify results

 

A single user request may trigger multiple model calls. For applications serving thousands or millions of users, even a small difference in cost per request can have a major impact.

 

Architects must always ask: "Does this application truly require the most expensive model, or can a smaller model achieve the same business goal?"

 

c. Speed: How Quickly Does It Respond?

Latency is another critical factor. Users expect AI applications to feel interactive.

 

A customer asking, "What is my account balance?" expects an answer almost immediately.

 

A delay of several seconds may be acceptable for:

 

·      Research assistance

·      Document analysis

·      Code generation

 

but unacceptable for:

·      Customer support chat

·      Voice assistants

·      Real-time applications

 

Smaller models are usually faster because they require fewer computational resources. Therefore, many production systems use different models for different tasks:

 

·      A fast model for simple conversations.

·      A powerful model for complex reasoning.

 

d. Privacy: Where Does the Data Go?

Privacy is becoming one of the most important architectural considerations. Many enterprise applications handle sensitive information:

 

·      Customer data

·      Financial records

·      Healthcare information

·      Internal company documents

·      Intellectual property

 

The key question becomes, "Can this data be sent to an external AI provider?"

 

For some organizations, the answer is no. In these cases, companies may choose to:

 

·      Deploy open-source models internally

·      Run models inside their own cloud environment

·      Maintain complete control over data processing

 

The trade-off is that the organization becomes responsible for:

 

·      Infrastructure

·      Model deployment

·      Scaling

·      Monitoring

·      Security updates

·      Performance optimization

 

9.2 The Architect's Decision Matrix

There is no universally "best" Foundation Model. The right choice depends on the application.

Requirement	Preferred Choice
Maximum reasoning ability	Large, powerful commercial models
Low latency and high volume	Smaller, optimized models
Strict data privacy	Self-hosted open-source models
Experimental applications	Cost-effective models
Mission-critical decisions	Highest capability models with validation

 

How Architects Typically Build Real Systems

In production environments, architects rarely choose only one model. A common pattern is a multi-model architecture.

 

For example:

·      A simple FAQ question may go to a fast, inexpensive model.

·      A complex business analysis request may go to a powerful reasoning model.

·      A sensitive financial document may go to a private deployment.

 

This approach provides the best balance between performance, cost, and security.

 

In summary, choosing a Foundation Model is not about selecting the model with the highest intelligence score. It is an architectural decision based on four key trade-offs:

 

·      Capability: How well can the model reason and solve problems?

·      Cost: How much does each interaction cost at scale?

·      Speed: How quickly can it respond?

·      Privacy: Can your data be processed externally?

 

A good AI architect understands these trade-offs and selects the model that best fits the application's business requirements. The goal is not the smartest possible AI. The goal is the right AI for the right problem.

 

10. The Brain Alone Isn't an AI Agent

Throughout this article, we've explored the most important component of an AI agent, its brain. We've learned what a Foundation Model is, how it learns from massive datasets, why next-token prediction leads to intelligent behaviour, how prompts shape its reasoning, and how architects choose the right model for different applications.

 

At this point, it's easy to think, If Foundation Models are so powerful, are they the AI agent?. The answer is no. This is one of the most important architectural concepts to understand. A Foundation Model and an AI Agent are not the same thing. A Foundation Model is the brain of the system. An AI agent is the complete intelligent system.

 

A Foundation Model is exceptionally good at tasks such as:

 

·      Understanding natural language

·      Answering questions

·      Generating text

·      Writing code

·      Summarizing documents

·      Translating languages

·      Reasoning through problems

·      Creating plans

 

These capabilities make it an excellent reasoning engine. But on its own, the model has significant limitations. For example, it cannot automatically:

 

·      Remember what you discussed yesterday

·      Search your company's internal knowledge base

·      Check today's weather

·      Read a PDF from your computer

·      Send an email

·      Book a flight

·      Execute a SQL query

·      Call an external API

 

It can explain how to do these things, but it cannot perform them by itself. This is because a Foundation Model is designed to reason, not to interact with the outside world.

 

10.1 An AI Agent Does Much More

An AI agent takes the reasoning capabilities of a Foundation Model and surrounds it with additional components that enable it to operate like an intelligent assistant. Instead of simply generating text, an AI agent can:

 

·      Understand user requests

·      Remember previous conversations

·      Break large goals into smaller tasks

·      Decide which tools to use

·      Retrieve information from external systems

·      Execute actions

·      Evaluate the results

·      Continue until the goal is completed

 

In other words, an AI agent doesn't just answer questions. It works toward achieving a goal.

 

11. Summary

In this article, we explored the brain of an AI agent, the Foundation Model and discovered why it is the most critical component in any intelligent AI system.

 

We learned that every modern AI agent begins with a Foundation Model, and that Large Language Models (LLMs) are simply one category of foundation models specialized for understanding and generating human language. While today's AI agents are predominantly powered by LLMs, the future is increasingly moving toward multimodal foundation models that can reason across text, images, audio, and video.

 

We also examined how foundation models are trained on enormous and diverse datasets. Rather than memorizing facts, they learn statistical patterns and relationships across language, code, and knowledge. At their core, they perform a remarkably simple task, predicting the next token. Yet, when this process is repeated trillions of times using powerful transformer architectures and massive-scale training, it gives rise to impressive capabilities such as reasoning, planning, inference, and code generation.

 

Next, we saw how prompt engineering transforms a pretrained model into a task-oriented assistant. Through system prompts, developer prompts, and user prompts, architects can control how a model behaves, what constraints it follows, and how it approaches complex problems. We also introduced reasoning patterns such as Chain of Thought and ReAct, which enable AI agents to think step by step, use external tools, observe results, and make better decisions.

 

From an architectural perspective, we discussed why choosing a Foundation Model is not about selecting the smartest model available. Instead, architects must carefully balance four key trade-offs: capability, cost, speed, and privacy. The right model depends on the application's goals, operational constraints, and business requirements not simply benchmark scores.

 

Finally, we arrived at one of the most important architectural insights of this series, a Foundation Model is the brain of an AI agent, but the brain alone is not an AI agent.

 

A true AI agent emerges only when the Foundation Model works together with components such as memory, perception, planning, tools, knowledge, orchestration, and action. Together, these components transform raw intelligence into a complete system that can understand its environment, reason about goals, interact with external systems, and accomplish meaningful tasks autonomously.

 

In the next article, we'll continue our journey through the anatomy of an AI agent by exploring the next essential component and understanding how it contributes to building intelligent, production ready AI systems.

 

Previous                                                    Next                                                    Home