Maanas Arora, January 19, 2026
Background
Today’s AI technologies are based largely on machine learning, a technique through which computers learn patterns from large amounts of data. A particularly important form of machine learning is deep neural networks.1 These machine learning models are partly inspired by the structure of the human brain and can perform more advanced tasks. For example, a traditional machine learning model may learn patterns from housing data to predict house prices from its size and location, whereas a modern neural network can learn from less conventional data such as flower images to identify flowers in new images. Neural network technology has transformed the AI space, with applications ranging from scanning images of handwriting2 to augmenting biologists’ search for proteins3, and more recently used in language models such as LLMs.
Large language models (LLMs) are AI models that use neural networks to identify patterns in text.4 The goal is to teach them to predict the most likely next word following a piece of text in human language. For example, given a partial sentence such as “My name is…”, an LLM can produce a list of words that are most likely to follow, such as “John”, and a number for each of them, such as 1.2%, describing the chance of it being the next word. By repeating this process on their own output, LLMs can string words together, thus producing longer passages of text. For example, the LLM can use its output above, “My name is John”, and continue to predict the following words to produce a sentence such as “My name is John and I am a teacher.” Because LLMs are based on probabilities, they cannot predict text with certainty and instead excel at text generation, hence LLMs falling under the category of generative AI.
Hardware
Neural networks and LLMs require much more computer power and memory than traditional machine learning. While neural networks can be small or large, for most recent use cases, they produce millions or even billions of numbers for each prediction in order to find increasingly complex patterns in the data. As a result, they must usually run on specialized hardware such as graphics processing units (GPUs) to run efficiently. Since neural networks require so much processing to reach their predictions, it is not possible for humans to accurately inspect how they come to specific outputs, making errors difficult to diagnose and fix reliably.5
Training
The life-cycle of an LLM can be divided into four phases: pre-training, training, validation, and deployment. The phases do not always happen in order, but they provide a general picture of the process.
Pre-training involves training an LLM to predict the next word from very large amounts of text data, usually to the order of billions or trillions of words, often sourced from the internet, books, or other media sources. The goal for this stage is to learn relatively general patterns from this data about the use and order of words in text data, and therefore the relationships between words. For example, a model correlates the words “Tokyo” and “Toronto” because they are both found in similar contexts as cities. It learns that different kinds of words, e.g. nouns, verbs, and articles, are placed in different order in a sentence. Hence it can be said to gain a general understanding of language patterns from the text data. As it must predict text created in real-world contexts, it also encodes facts and information about the real world, for example such as following “The capital of Canada is…” with “Ottawa’.
Training or fine-tuning an LLM takes these learned patterns and refines them to better apply to a specific task. During pre-training an LLM may learn to predict text from any section of the internet, and during fine-tuning the LLM can be further adapted to answer simple multiple-choice questions about science. Fine-tuning usually requires much less text data than pre-training because the model already contains general information about language through pre-training and that existing information is updated to better reflect the patterns in the specific kind of text used to fine-tune.
To validate the performance of an LLM after pre-training or fine-tuning, specific datasets can be used. For example, researchers can gather a set of multiple-choice or one-word questions and answers on various topics, and ask LLMs to answer those questions without having access to the answer. Depending on how many answers given by the LLM match this “ground truth” dataset, metrics such as accuracy (percentage of correct answers) can be calculated.6 These metrics can be used to further improve training. Notably, many of these datasets, or benchmarks, are focused on simple tasks such as answering factual questions, since it is difficult to clearly define what makes an LLM “effective” for tasks such as generating passages of text.
Reinforcement Learning
LLMs can also be refined using feedback received from human users in a process called Reinforcement Learning from Human Feedback (RLHF). RLHF has been used extensively on most commercially active LLM products to improve generated responses.7 Researchers, supervised workers, or users will indicate their preferences for some responses over others, and the numbers composing the LLM adapt to generate responses that reflect this preference. For example, after querying a LLM, a user may indicate that a response is helpful, which is used to adapt the LLM to produce results that are more similar to user preferences in the future.
Reinforcement learning is effective at adapting to feedback and does not consider the factors which lead to user feedback. Experts perceive this to be an important factor leading to AI sycophancy i.e., when LLMs generate responses that the user finds pleasant or agreeable, even if inaccurate.8
References
1. Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep Learning. Adaptive Computation and Machine Learning. The MIT press, 2016.
2. Jumper, John, Richard Evans, Alexander Pritzel, et al. “Highly Accurate Protein Structure Prediction with AlphaFold.” Nature 596, no. 7873 (2021): 583–89. https://doi.org/10.1038/s41586-021-03819-2.
3. Lecun Y., L. Bottou, Y. Bengio, and P. Haffner. “Gradient-Based Learning Applied to Document Recognition.” Proceedings of the IEEE 86, no. 11 (1998): 2278–324. https://doi.org/10.1109/5.726791.
4. Naveed, Humza, Asad Ullah Khan, Shi Qiu, et al. “A Comprehensive Overview of Large Language Models.” ACM Transactions on Intelligent Systems and Technology 16, no. 5 (2025): 1–72. https://doi.org/10.1145/3744746.
5. Sharma, Mrinank, Meg Tong, Tomasz Korbak, et al. “Towards Understanding Sycophancy in Language Models.” Version 4. Preprint, arXiv, 2023. https://doi.org/10.48550/ARXIV.2310.13548.
6. Wang, Alex, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. “GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding.” Version 3. Preprint, arXiv, 2018. https://doi.org/10.48550/ARXIV.1804.07461.
7. Zhang, Yu, Peter Tino, Ales Leonardis, and Ke Tang. “A Survey on Neural Network Interpretability.” IEEE Transactions on Emerging Topics in Computational Intelligence 5, no. 5 (2021): 726–42. https://doi.org/10.1109/TETCI.2021.3100641.
8. Ziegler, Daniel M., Nisan Stiennon, Jeffrey Wu, et al. “Fine-Tuning Language Models from Human Preferences.” Version 2. Preprint, arXiv, 2019. https://doi.org/10.48550/ARXIV.1909.08593.
