Large language models have transformed how we interact with technology, enabling machines to understand and generate human-like text with remarkable fluency. These powerful AI systems underpin tools like ChatGPT and are reshaping education, business, and creative work. In this lesson, you will learn what large language models are, how they work, and how to use them effectively in everyday tasks.
Large Language Models: What You’ll Learn
This lesson introduces you to the foundational concepts behind large language models, from their origins in artificial intelligence research to the neural networks that power them today. By the end, you will understand how large language models process language and generate responses.
You will also explore practical concepts such as tokens, context windows, and temperature settings. These ideas will help you get better results when working with large language models in real-world scenarios.
What Is Artificial Intelligence?
Artificial intelligence, or AI, is the broad field of computer science focused on building systems that can perform tasks normally requiring human intelligence. These tasks include recognising images, understanding speech, making decisions, and processing natural language. The concept of AI has existed for decades, but recent breakthroughs in computing power and data availability have accelerated progress enormously.
At its core, AI is not a single technology but a collection of approaches. Some AI systems use explicit rules programmed by humans, while others learn patterns from large datasets. This second category, known as machine learning, is where large language models live. Machine learning algorithms improve their performance automatically as they are exposed to more data.
Within machine learning, a subfield called deep learning uses artificial neural networks with many layers. These deep networks can learn extremely complex patterns, which is why they excel at tasks like image recognition and language understanding. Large language models are built using deep learning architectures specifically designed to handle text.
From Narrow AI to General Intelligence
It is important to distinguish between narrow AI and artificial general intelligence. Narrow AI, which includes all current large language models, is designed to excel at specific tasks. A language model can write essays or answer questions, but it cannot drive a car or cook a meal without being specifically trained for those tasks.
Artificial general intelligence, by contrast, would match or exceed human capabilities across all domains. This remains a theoretical goal. Today’s large language models represent some of the most advanced narrow AI systems ever built, but they are still specialised tools rather than thinking beings.
What Are Large Language Models?
Large language models are a type of AI system trained to understand and generate human language. They achieve this by analysing vast amounts of text data and learning the statistical relationships between words, phrases, and sentences. When you type a prompt into ChatGPT, the large language model behind it predicts what text should come next based on patterns it learned during training.
The word “large” in large language models refers to both the size of the training dataset and the number of parameters in the model. Parameters are the internal values the model adjusts during training to improve its predictions. Modern large language models contain billions or even trillions of parameters, allowing them to capture nuanced patterns in language.
Large language models do not truly “understand” language the way humans do. Instead, they recognise statistical regularities. For example, after seeing the phrase “once upon a” millions of times during training, the model learns that “time” is a likely next word. This predictive capability, scaled across enormous datasets, produces surprisingly coherent and useful output.
How Large Language Models Differ from Traditional Software
Traditional software follows explicit instructions written by programmers. If you want a program to sort a list, you write step-by-step logic for sorting. Large language models work differently. Instead of being told the rules, they discover patterns from data. This makes them far more flexible but also less predictable.
This distinction matters because it explains both the power and the limitations of large language models. They can handle open-ended requests that would be impossible to program with rigid rules, but they can also make mistakes that rule-based systems would avoid.
Neural Networks: The Engine Behind Large Language Models
Neural networks are the computational structures that power large language models. Inspired loosely by the human brain, a neural network consists of layers of interconnected nodes, often called neurons. Each connection between nodes has a weight, which determines how strongly one neuron influences another. During training, these weights are adjusted so the network produces better outputs.
A typical neural network has an input layer, one or more hidden layers, and an output layer. Data enters through the input layer, flows through the hidden layers where transformations occur, and exits through the output layer as a prediction or classification. The depth of hidden layers is what gives deep learning its name.
Large language models use a specific neural network architecture called the transformer. Introduced in a landmark 2017 paper, transformers revolutionised natural language processing by introducing a mechanism called attention. Attention allows the model to weigh the importance of different words in a sentence, capturing context and relationships that earlier architectures missed.
The Transformer Architecture
Transformers process text in parallel rather than sequentially, which makes training far more efficient. The self-attention mechanism lets each token in a sequence attend to every other token, so the model can understand how words relate regardless of their position. This is why large language models can handle long-range dependencies in text.
The transformer’s impact has been enormous. Virtually every major large language model today, from OpenAI’s GPT series to Google’s Gemini, is built on transformer-based architecture. Understanding this foundation helps you appreciate why large language models are so capable at generating contextually relevant text.
Training vs Inference: Two Critical Phases
Every large language model goes through two distinct phases: training and inference. Training is the process of building the model. During training, the model is exposed to enormous datasets containing text from books, articles, websites, and other sources. The model’s parameters are adjusted through repeated iterations so that it becomes better at predicting the next word in a sequence.
Training a large language model requires massive computational resources. It can take weeks or months, running on clusters of specialised hardware like GPUs. The cost can reach millions of dollars for the largest models. Once training is complete, the model’s parameters are fixed, and it is ready for the next phase.
Inference is the phase where the trained model is actually used. When you send a prompt to ChatGPT, you are performing inference. The model takes your input, processes it through its neural network, and generates a response. Inference is much faster and cheaper than training, but it still requires significant computing power.
Why This Distinction Matters
Understanding the difference between training and inference helps you set realistic expectations. A large language model cannot learn new information during inference. Its knowledge is frozen at the point training ended. This is why models can have outdated information and why fine-tuning, a lighter version of training, is sometimes used to update them.
This also explains why large language models sometimes produce confident but incorrect answers. They are not looking things up in real time; they are generating responses based on patterns learned during training.
Tokens: How Large Language Models Read Text
Large language models do not process text word by word. Instead, they break text into smaller units called tokens. A token can be a whole word, a part of a word, or even a single character. For example, the word “hamburger” might be split into tokens like “ham,” “bur,” and “ger.” Common words are usually single tokens, while rare or complex words get split.
Tokenisation affects both cost and performance. When you use an API to access large language models, you are typically charged based on the number of tokens processed. Understanding tokens helps you write more efficient prompts and manage costs.
import tiktoken
# Load the tokenizer used by GPT models
enc = tiktoken.encoding_for_model("gpt-4")
# See how text becomes tokens
text = "Large language models are fascinating."
tokens = enc.encode(text)
print(f"Text: {text}")
print(f"Tokens: {tokens}")
print(f"Token count: {len(tokens)}")
# Decode tokens back to text
decoded = enc.decode(tokens)
print(f"Decoded: {decoded}")
You can see from this example that a single sentence becomes multiple tokens. The mapping between tokens and text is not always intuitive, which is why token-awareness is a valuable skill when working with large language models.
Approximating Token Counts
A useful rule of thumb is that one token corresponds to roughly four characters of English text, or about three-quarters of a word. A 1,000-word essay would contain approximately 1,300 tokens. This estimate helps you gauge how long your prompts and expected responses might be.
Token limits matter because every large language model has a maximum number of tokens it can process in a single request. This leads us to the next critical concept: context windows.
Context Windows in Large Language Models
The context window is the maximum amount of text a large language model can consider in a single interaction. It includes both your input prompt and the model’s generated response. If your conversation exceeds the context window, earlier messages may be truncated or forgotten.
Think of the context window as the model’s short-term memory. Within this window, the model can reference everything that has been said. But once the conversation grows beyond the limit, the earliest content falls out of scope. This is why providing relevant context upfront, rather than burying it in a long conversation, leads to better results.
Different large language models have different context window sizes. Early models could handle only a few thousand tokens. Modern models support context windows of tens of thousands or even millions of tokens, enabling analysis of long documents, code repositories, and extended conversations.
Managing Context Effectively
To get the best results from large language models, keep your context window focused on relevant information. Avoid pasting unnecessarily long documents when a concise excerpt will do. If you need to reference specific facts, include them clearly in your prompt rather than hoping the model will recall them.
This strategy is particularly important for tasks like document summarisation, code debugging, or multi-step reasoning, where relevant details can easily get lost if the context window is overloaded with noise.
Temperature: Controlling Creativity in Large Language Models
Temperature is a parameter that controls how random or deterministic a large language model’s output is. It works by adjusting the probability distribution over possible next tokens. A low temperature makes the model more likely to choose the highest-probability token, producing focused and consistent output. A high temperature introduces more randomness, leading to creative but less predictable results.
Understanding temperature helps you fine-tune the behaviour of large language models for different tasks. A temperature of zero means the model always picks the most likely next token, making its responses deterministic. This is ideal for factual tasks, coding, and analytical work where consistency matters.
Temperature guide for large language models:
0.0 — Fully deterministic; best for code, math, factual Q&A
0.3 — Low randomness; good for summaries, structured output
0.7 — Balanced; good default for general conversation
1.0 — More creative; good for brainstorming, storytelling
1.5+ — Highly random; experimental, often incoherent
Choosing the right temperature depends on your goal. For technical work with large language models, a low temperature ensures reliable, repeatable answers. For creative writing or idea generation, raising the temperature can produce more varied and interesting results.
Practical Temperature Tips
Most users never need to adjust temperature manually when using ChatGPT, as sensible defaults are built in. However, if you access large language models through an API, experimenting with temperature can significantly improve results for specific use cases.
Start with the default temperature and adjust gradually. Small changes can have noticeable effects on output quality and consistency, especially in edge cases or unusual prompts.
How ChatGPT Works
ChatGPT is an application built on top of a large language model. When you type a message, your text is tokenised and sent to the model. The model processes your tokens through its transformer layers, using attention to understand context and intent. It then generates a response token by token, with each new token influenced by everything that came before it.
The model does not plan its full response in advance. It generates one token at a time, predicting the most likely continuation at each step. This autoregressive process continues until the model produces a special stop token or reaches a length limit. Despite this incremental approach, the results often appear coherent and well-structured.
ChatGPT also includes additional training beyond the base large language model. A process called reinforcement learning from human feedback, or RLHF, helps align the model’s responses with human preferences. This makes ChatGPT more helpful, harmless, and honest than the raw model alone.
The Full Pipeline from Prompt to Response
Behind the scenes, ChatGPT applies a system prompt that shapes its behaviour before your message is ever processed. It may also retrieve relevant information, format your input, and apply safety filters. All of this happens in seconds, giving you the impression of a seamless conversation with a knowledgeable assistant powered by large language models.
For a deeper dive, you can explore the official OpenAI documentation on GPT models, which covers these concepts in greater technical detail.
A Worked Example: From Prompt to Token
Let us trace exactly what happens when you send a prompt to a large language model. Imagine you type the following message into ChatGPT: “Explain what a neural network is in one paragraph.” This simple sentence triggers a sophisticated pipeline that transforms your words into a meaningful response.
First, the text is tokenised. The system breaks “Explain what a neural network is in one paragraph” into a sequence of numeric tokens. Each token maps to an entry in the model’s vocabulary, which typically contains tens of thousands of entries. The word “Explain” might be a single token, while “neural” could be split depending on the model’s vocabulary.
Next, these tokens are converted into high-dimensional vectors called embeddings. Each embedding captures semantic information about the token, so similar words have similar vectors. The transformer layers then process these embeddings, using self-attention to understand how each token relates to the others. The model builds a contextual understanding of your entire prompt.
Now the model begins generating output. It predicts the most probable next token given everything it has processed so far, including your prompt and any tokens it has already generated. This predicted token is added to the sequence, and the model repeats the process. Each new token is chosen from a probability distribution shaped by the model’s training and the current temperature setting.
If temperature is low, the model consistently selects the highest-probability token, producing a clear, direct explanation of neural networks. If temperature is higher, the model might take more creative liberties, potentially using analogies or unconventional language. The generation continues token by token until the model emits a stop signal or reaches the maximum response length.
Finally, the generated tokens are decoded back into human-readable text and displayed in your chat interface. This entire process, from tokenisation to decoding, typically takes just a few seconds for a paragraph-length response. Yet within that brief window, the large language model has performed billions of mathematical operations to produce output that reads as if a human wrote it.
Large Language Models: Common Mistakes to Avoid
Working with large language models is powerful, but certain pitfalls can undermine your results and lead to frustration or misinformation.
- Treating output as fact. Large language models can produce convincing but incorrect information. Always verify important claims with trusted sources before acting on them.
- Ignoring context limits. Stuffing too much text into a prompt can cause the model to miss key details. Keep your inputs focused and relevant.
- Overcomplicating prompts. Long, convoluted instructions often confuse the model. Clear, direct prompts yield better results from large language models.
- Expecting real-time knowledge. Models are trained on data up to a certain point. They cannot access current events unless connected to live data sources.
Large Language Models: Best Practices
- Write clear, specific prompts with all necessary context for better results.
- Start simple, then iterate based on the quality of responses you receive.
- Use low temperature for factual tasks and higher values for creative work.
- Break complex requests into smaller steps the model can handle sequentially.
- Always verify critical information from large language models against reliable sources.
Large Language Models: Frequently Asked Questions
What is a large language model?
It is a neural network trained on massive amounts of text to predict and generate language. Models like the one behind ChatGPT learn statistical patterns well enough to write, summarize, and answer questions in natural language.
How do tokens and the context window affect what it can process?
Text is broken into tokens before processing, and every model has a fixed context window limiting how many tokens it can consider at once. Once a conversation exceeds that window, earlier content starts falling out of view.
What does the temperature setting control?
Temperature controls how much randomness goes into generated text. Lower values produce focused, predictable output suited to factual tasks, while higher values add variety and creativity at the cost of consistency.
Why can these models produce confidently wrong answers?
They generate the statistically likely next words, not verified facts, so they can state incorrect information with the same fluent confidence as correct information. Always verify important outputs independently rather than trusting tone alone.
How is ChatGPT related to this technology?
ChatGPT is a product built on top of a large language model, adding a chat interface, memory, and safety layers around the underlying model so people can interact with it conversationally rather than through raw API calls.
Large language models represent one of the most significant advances in artificial intelligence, transforming how we create and interact with text. By understanding how these models work, you can use them more effectively and avoid common pitfalls.