Robot teaching students (created using Stable Diffusion)

If you're trying to catch up on AI terminology and don't know where to start, here's a glossary of the terms that come up most often. I pulled this together from the NYT's AI glossary and my own reading.

Large Language Model (LLM)

A type of neural network trained on massive amounts of text from the internet. LLMs learn to write, hold conversations, and generate code. They work by predicting the next word in a sequence, but along the way they pick up abilities that surprised even the people who built them.

Generative AI

Technology that creates new content (text, images, video, code) by learning patterns from training data. ChatGPT generates text; DALL-E and Midjourney generate images.

Transformer Model

A neural network architecture that processes an entire sentence at once rather than word by word. It uses "self-attention" to figure out which words in a sentence are most relevant to understanding the meaning. This is the architecture behind GPT and most modern language models.

Parameters

The numeric values inside a language model that determine how it behaves. Think of them as the knobs the model adjusts during training to get better at predicting text. Modern models like GPT-4 have hundreds of billions of parameters.

Reinforcement Learning

A training method where the model learns through trial and error, receiving rewards or penalties based on its performance. When combined with human feedback (RLHF), humans rate the model's outputs, and the model adjusts to produce better responses.

Hallucination

When an LLM generates something that sounds confident but is factually wrong, irrelevant, or nonsensical. This happens because the model is predicting plausible-sounding text, not checking facts.

Bias

Errors in a model's output that reflect biases in its training data. For example, a model might associate certain jobs with a particular gender or race because its training data contained those patterns.

Anthropomorphism

The tendency to treat AI chatbots as if they have feelings, intentions, or consciousness. When a chatbot responds politely, people sometimes assume it's being kind. It's not. It's predicting text.

Natural Language Processing

The techniques used to make computers understand and generate human language. This includes text classification, sentiment analysis, translation, and summarization. NLP uses a mix of machine learning, statistical models, and linguistic rules.

Emergent behavior

Capabilities that appear in a model that weren't explicitly trained for. For instance, an LLM trained on code repositories can write new code, and models trained on general text turn out to be able to write poetry or solve math problems.

Alignment

The ongoing effort to make sure AI systems behave in ways that match the values and goals set by their creators. This is one of the harder unsolved problems in AI.

Multimodal systems

AI systems that can process more than just text. They handle images, audio, video, and other inputs. GPT-4 with vision is an example.

Artificial General Intelligence (AGI)

A hypothetical AI system that can do anything a human mind can do. We're not there yet, and there's active debate about what "getting there" would even mean.

---

References

• Artifical Intelligence Glossary - https://www.nytimes.com/article/ai-artificial-intelligence-glossary.html