How Does Al Actually Work ? Tokens, Prompts, Context, and RAG Explained. From Basics To Advance
Most people think AI means tools like ChatGPT, Claude, or image generators.
That’s only a small part of the picture.
AI is not a single tool—it’s a system made up of multiple layers: tokens, models, prompts, context, retrieval, and agents.
In this article, we’ll break down how AI actually works in simple terms, without jargon.
I remember it was the beginning of 2023.
Something shifted almost overnight.
Suddenly, everyone was talking about AI.
My social media feeds were flooded. News headlines couldn’t get enough of it. Even people far outside the tech world—people who had never cared about code, algorithms, or data—were casually dropping the term “AI” into conversations like it had always been part of their vocabulary.
It felt like a wave. Fast, loud, and impossible to ignore.
But here’s the interesting part—while everyone was talking about AI, very few actually understood what it was.
Fast forward to today, and not much has changed.
AI is still everywhere. It dominates discussions, drives trends, and shapes industries. Yet for most people, AI has been reduced to a handful of tools—chatbots like ChatGPT, image generators, or assistants like Claude and Gemini.
Some think AI is just about generating images. Others think it’s about writing emails or code.
That’s not wrong—but it’s not even close to the full picture.
AI is not a single tool. It’s not a product. It’s not even one technology.
It’s an umbrella—covering a vast landscape of ideas, systems, and approaches that have been evolving for decades, long before the recent hype cycle began.
In this article, we’re going to step back from the noise and break things down clearly. No jargon. No unnecessary complexity. Just a simple, structured understanding of what AI actually is, how it works, and why it matters far beyond the tools you see today.
The Big Umbrella - What is AI?
The best mental model I've come across is this: AI is an umbrella term. Everything else — Machine Learning, Deep Learning, Generative AI, LLMs, all of it — lives underneath that umbrella.
Starting from the first so
what is Artificial Intelligence?
Artificial Intelligence just means:
A machine doing something that normally requires human intelligence. That's it…
It's the broadest possible definition.
Chess-playing programs from the 1990s were AI. So is the spam filter in your email. So is ChatGPT. They're wildly different things, but they all fit under the umbrella.
The question is — what's inside it?
What is Machine Learning ?— Teaching by Example
The first thing under the umbrella is Machine Learning, and it changed everything about how we build AI systems.
Before ML, if you wanted a computer to do something intelligent, you had to tell it every rule explicitly(Rule based models).
Want it to detect spam? You write a rule: "If the email contains the word 'lottery', flag it." Then another rule. Then another… It was exhausting, brittle, and fell apart the moment spammers changed their tactics.
Machine Learning flipped this around. Instead of writing the rules, you show the machine thousands of examples and let it figure out the patterns itself. Show it ten thousand spam emails and ten thousand normal ones, and it learns — on its own — what distinguishes them.
This is what "learns from data" means. The learning isn't metaphorical; the model is literally adjusting its internal parameters based on what it sees, getting better with more examples.
There are three primary approaches to training AI systems.
Supervised learning works with labeled data—you’re essentially teaching the model by example: “this is spam, this is not.”
Unsupervised learning takes away those labels and asks the system to uncover hidden patterns on its own—“here’s the data, figure out what’s going on.”
And then there’s reinforcement learning, where the model learns through trial and error—making decisions, receiving feedback, and improving over time, much like how humans learn from rewards and mistakes.
Each approach has its strengths, and all three continue to power modern AI systems in different ways.
What is Deep Learning ? — Stacking the Layers
Machine Learning worked brilliantly for structured data — spreadsheets, transaction records, click patterns. But it struggled with messy, unstructured stuff like images, audio, and language. Those things don't fit neatly into rows and columns.
Deep Learning was the breakthrough that unlocked all of that.
The idea came from the brain (loosely). Neurons in the brain are connected in layers, and information flows through those layers, getting processed and refined at each step. Deep Learning uses artificial neural networks — layers of mathematical nodes — to do something similar.
The “deep” just means there are many layers. Early networks had two or three. Modern ones have hundreds or thousands. Each layer learns to recognize increasingly abstract features — the first layer of an image recognition network might detect edges, the next layer curves, the next layer shapes, and eventually something recognizable like “face” or “car.”
Deep Learning is why your phone can unlock with your face. Why Spotify knows what you want to hear next. Why translation tools got dramatically better around 2016. It was the engine powering a decade of AI progress before most people noticed AI was progressing.
What is Generative AI ? — The Shift From Recognizing to Creating
Machine Learning and Deep Learning were mostly about understanding things — classifying, predicting, recognizing. Generative AI does something different: it creates.
Text. Images. Code. Audio. Video. Given a prompt, it generates new content that didn’t exist before. This is the category that made AI visible to everyone, not just researchers and engineers. DALL-E generating images from text descriptions. GitHub Copilot writing code as you type. ChatGPT having a conversation.
The shift feels subtle but it’s profound. We went from “AI that tells you what’s in a photo” to “AI that makes the photo.” From “AI that detects spam” to “AI that writes your email.” The applications expanded by an order of magnitude overnight.
what is Multimodality ?
Each of the outputs like Text to text, Text to audio, Text to Video, Vision Language Model ( process image and text together) requires different own dedicated model.
That’s changing; models like GPT-5 and Gemini now handle text, images, audio, and video within a single architecture.
If your question is how?How does a text prompt like “a photo of a cat” actually guide an image model to generate the right image?
Then the ans is its cross-modal
what is Embedding?
An embedding converts content into a list of numbers that captures its meaning. Cross-modal embeddings do this across different content types, text and images, placing them on the same map.
CLIP, developed by OpenAI, is the most well-known example. It places both photo of a cat" (text) and an actual
What is LLMs — The Engine Under the Hood
When people say “AI” today, they usually mean a Large Language Model. This is the specific type of deep learning model that underpins ChatGPT, Claude, Gemini, and most of the AI tools you actually interact with daily.
LLMs are trained on enormous amounts of text — books, websites, code, conversations — and they learn to predict what word (technically what token) comes next in a sequence. That sounds almost trivially simple, and yet it produces systems that can reason, explain, summarize, translate, and hold coherent conversations.
The “large” part matters. These models have billions of parameters — internal weights that encode patterns from all that training data. The scale is what makes them surprisingly capable. Small models are fine for narrow tasks. Large models generalize.
What an LLM actually does when you send it a message is predict the most likely next token, then the one after that, thousands of times in sequence. The result reads like something a thoughtful person wrote. Because it was trained on things thoughtful people wrote.
What is Tokens ? — The Atomic Unit
Speaking of tokens — this word comes up constantly and deserves thirty seconds.
A token is roughly a word, or a piece of a word. “Unbelievable” might be two or three tokens. “The” is one. Code, punctuation, spaces — they’re all tokens too.
LLMs don’t process letters and they don’t process sentences. They process tokens. When you hear “this model supports a 128K context window,” that means it can process about 128,000 tokens — roughly 100,000 words — at once. Tokens are also the unit pricing is based on when you use an API. Now when someone says “you’re running low on tokens,” you know exactly what they mean.
What is Tokenizer ?
Tokens are just the beginning. They’re the raw input. But what happens next is where things get interesting.
Before a model can “understand” anything, those tokens have to be converted into numbers. That’s the job of a tokenizer. It breaks text into tokens and maps each one to a unique numerical ID. To a machine, everything is math—so this step is non-negotiable.
But IDs alone aren’t enough. The model needs meaning, not just labels. That’s where embeddings come in.
What is Embedding ?
The numbers are not enough to mean or tell a story . So the model takes another step
It begins to give those numbers meaning. Each token is stretched into a vector—a bundle of numbers that quietly captures its context, its relationships, its subtle shades of meaning. Suddenly, “king” isn’t just a word—it sits close to “queen,” not because someone told it to, but because the patterns say so. “Apple” might drift toward fruit in one moment, and toward technology in another.
So when you type a sentence, what really happens is a quiet journey:
your words are broken apart, turned into numbers, shaped into meaning, placed on a map, and guided by an intricate web of learned connections—until, finally, a response emerges.
And once you see it this way, it stops feeling like magic.
It starts to feel like a story unfolding beneath the surface.
What is Parameters ?
Parameters are the internal numerical values that the model learns during training. They determine how strongly one piece of information is connected to another—how patterns are recognized, how decisions are made, and ultimately, how outputs are generated.
You can think of them as millions or billions of tiny adjustable knobs inside the system. Training a model is essentially the process of tuning these knobs so that the model produces useful and accurate results.
But parameters are not knowledge in the human sense.
The model doesn't store facts like a database. Instead, it encodes patterns across these parameters-statistical relationships learned from vast amounts of data. That's why it can generate fluent responses, yet still make mistakes or hallucinate.
What is Prompt Engineering — The Craft of Asking Well
The quality of an AI model’s output is heavily dependent on how you phrase the input.
Same model. Same capabilities. Different prompt → different result.
This is where prompt engineering comes in.
It’s the practice of structuring inputs in a way that guides the model toward better, more accurate, and more useful responses. Despite sounding informal, it has a measurable impact. For example, adding instructions like “think step by step” can significantly improve reasoning performance.
Effective prompts often include:
Clear instructions
Defined roles (e.g., “act as a software engineer”)
Output format constraints
Context and boundaries (what to include or avoid)
Give Example (Few Shot Technique)
These elements reduce ambiguity and help the model align its response with your intent.
For most users, prompt engineering is the highest-leverage skill when working with AI. You don’t need to understand the underlying architecture or training process. The ability to ask precise, structured questions is often enough to unlock significantly better results.
What is Context Engineering ?— The Bigger Picture
Prompt Engineering is what an individual does in a single conversation. Context Engineering is what engineers do when building AI-powered products.
At the application level, you’re not just writing prompts — you’re managing everything the model can see at once: the system instructions, the conversation history, external data sources, the outputs of tools it can call. You’re deciding what information gets included, in what order, in what format, and what gets left out when you hit the context limit.
It’s the difference between having a conversation with an AI and building something with AI. Context Engineering is the craft of making sure the model always has exactly the right information at the right time to do its job well.
A simple chat interaction relies on prompting. A reliable AI product depends on context engineering—ensuring the model consistently has the right information, in the right structure, at the right moment.
In practice, this often involves techniques like retrieval (RAG), memory management, and tool orchestration.
What is RAG ? — Giving AI Access to What It Doesn’t Know
Large Language Models have a built-in constraint: their knowledge is fixed at the time of training.
They don’t inherently know recent events, private data, or anything outside what they were trained on.
Retrieval-Augmented Generation (RAG) addresses this limitation.
Instead of relying only on stored knowledge, a RAG system retrieves relevant information from external sources in real time—documents, databases, APIs—and injects it into the model’s context before generating a response.
The model doesn’t “learn” this new data permanently. It simply uses it for that specific query.
A useful way to think about it:
A standard LLM is like a closed-book exam—it answers from memory.
A RAG system is like an open-book exam—it looks up the most relevant material before answering.
This shift is critical. It allows AI systems to:
Answer questions about proprietary company data
Reference up-to-date information
Ground responses in verifiable sources
In production systems, RAG typically follows a pipeline:
query → retrieve relevant chunks → inject into context → generate response
This is how modern AI assistants stay relevant and useful beyond their training data.
Most users don’t see it—but RAG is quietly powering a large portion of real-world AI applications today.
What is Fine Tuning ?
Different industry's demand different kind of specific model to the task. For example a Medical model should be well trained to provide suggestions same goes for financial model.
Fine-tuning is the process where we take the foundation model and train it or scale it on a specific task or datasets to specialize its behavior.
What is Hallucination ?— When AI Isn’t Correct
One of the most important limitations of AI systems is something called hallucination.
A hallucination occurs when a model generates information that sounds confident and coherent—but is factually incorrect, misleading, or completely fabricated.
This isn’t a bug in the traditional sense. It’s a byproduct of how these models work.
LLMs don’t retrieve facts like a database. They generate text based on patterns learned during training. Their goal is to produce the most likely sequence of words—not necessarily the most accurate one.
That’s why they can:
Invent sources that don’t exist
State incorrect facts with high confidence
Fill gaps in knowledge with plausible-sounding guesses
This is also why grounding becomes critical.
Techniques like RAG (Retrieval-Augmented Generation) help reduce hallucination by injecting real, relevant data into the model’s context before it responds. The model is no longer guessing—it’s working with actual information.
But hallucination never fully disappears. It can only be managed.
In practice, this means:
Verifying important outputs
Providing clear context and constraints
Using external data sources when accuracy matters
AI is powerful—but it is not a source of truth by default.
Understanding this distinction is what separates casual users from serious practitioners.
What is an Autoregressive Model?
Most Models we use ( as of 2026 April) most of them are Autoregressive model.
That simply means they generate text one token at a time, always based on what came before.
At each step, the model looks at the existing sequence and predicts the most likely next token. Then it adds that token to the sequence and repeats the process-again and again-until the response is complete.
What is Diffusion Model ?
Instead of producing one piece at a time, they start with pure random noise and gradually remove it over many steps until a coherent image (or video) emerges. During training, the model sees millions of images with noise added at various levels and learns to predict what the clean version looks like. At generation time, it reverses the process step by step, typically 20 to 50 steps, removing a little noise each time until a clear image forms. This is how Mid-journey, Stable Diffusion, DALL-E, and Sora work.
What is an Agentic AI ?— From Responding to Acting
So far, we’ve looked at AI as something that responds—give it input, get output.
Agentic AI changes that model entirely.
An AI agent is not just generating text. It is taking actions to achieve a goal.
Instead of a single prompt-response cycle, an agent operates in a loop:
”Understand the task → plan steps → take action → observe results → adjust → repeat”
This allows it to handle multi-step, real-world problems.
For example, instead of asking:
“Summarize this article,”
You can assign a goal:
“Research this topic, compare sources, and generate a report.”
An agent can then:
Search for information (often using RAG)
Decide which data is relevant
Call tools (APIs, databases, code execution)
Refine its own outputs
Iterate until the task is complete
This is where concepts start to converge:
Prompt engineering defines behavior
Context engineering manages information
RAG provides up-to-date knowledge
Tools enable real-world interaction
Agentic AI sits on top of all of this.
It turns AI from a passive system into something closer to a digital operator—capable of reasoning, acting, and adapting toward a defined objective.
This is also where the risks increase.
More autonomy means more room for errors, unintended actions, and compounding mistakes. Designing reliable agents requires careful constraints, monitoring, and fallback mechanisms.
But despite the challenges, this is the direction AI is moving toward.
Not just systems that answer questions—
but systems that get things done.
Closing - Seeing the System, Not the surface
AI today is surrounded by noise. Tools, trends, headlines, hype.
But underneath all of that, there’s a structure—a system that is surprisingly consistent once you see it clearly.
At the lowest level, it’s tokens and probabilities.
Then come embeddings and latent space—how meaning is represented.
Then prompting and context—how behavior is shaped.
Then RAG—how knowledge is extended.
Then agents—how actions are executed.
What most people interact with is just the surface: a chat interface, an image generator, a quick answer.
What actually matters is everything beneath it.
Once you understand this stack, AI stops being mysterious. It becomes predictable. Controllable. Even strategically useful.
And that’s the real shift.
The advantage is no longer just in using AI—it’s in understanding how to use it well.
AI isn’t magic.
It’s a system.
The question is— are you using it casually?
or are you using it with intent?
Because that difference compounds fast….
Written by Amitt Saha
Software Engineer & Author
Focused on Technology & Thinking
