Embeddings: meaning as a map

Every time an AI tool finds the right paragraph in your files, suggests a related article, or understands that "my AC quit" and "air conditioner not working" are the same complaint, an embedding is doing the quiet work underneath. You never see one. You never type one. But once you picture what it is, a lot of AI behavior stops looking like magic and starts looking like geography.

This lesson opens that box with no math at all. You already know from the tokens lesson that a model reads text in small chunks called tokens. Embeddings are what happens to those chunks, and to whole passages, so the computer can compare them by meaning.

A list of numbers that stands for meaning

An embedding is a list of numbers that stands in for a piece of text. OpenAI's own documentation describes it exactly that way: a list of numbers, where the distance between two lists tells you how related the two pieces of text are. Small distance, closely related. Large distance, not much in common.

The easiest way to picture a list of numbers is as an address. Two numbers can place a dot on a flat map: how far across, how far up. Give every word its own dot, and you have a map of words. The trick of embeddings is that the dots are not scattered at random. Words that mean similar things land in the same neighborhood. Dog sits near puppy. Refund sits near cancel. King sits near queen.

Nobody hand-places those dots. The positions are learned. During training, a model reads an enormous amount of text and slowly nudges each word's address until words that show up in similar sentences end up close together. The map is a side effect of reading, not something anyone drew.

One more thing worth knowing up front: embeddings are not only for single words. A tool can make one list of numbers for a whole sentence, a paragraph, or a page. That single address sums up what the passage is about, which is what makes the search tricks later in this lesson possible.

Similar meanings land near each other

Here is a small, hand-drawn version of the idea. Twenty words, placed by hand to show how a real map behaves. Tap any word to light up its three closest neighbors. Then try hot, refund, and king, and read the note under each.

Notice what you did not have to do. You did not tell the map that puppies are young dogs, or that a deposit has something to do with a booking. The neighborhoods appear because those words keep company in real writing. People who write about puppies also write about dogs. People who write about refunds also write about cancellations. The map is a record of who hangs out with whom.

A British linguist, J. R. Firth, put it this way back in 1957: "You shall know a word by the company it keeps." Embeddings are that saying, turned into a machine.

Directions carry meaning too

Neighborhoods are only half the story. On a good map, the direction from one word to another can mean something.

The classic example comes from a 2013 research paper called "Efficient Estimation of Word Representations in Vector Space," which helped make word maps famous. Picture the walk from man to woman on the map. Now picture the walk from king to queen. On a well-trained map, those two walks point in roughly the same direction and go roughly the same distance. The same goes for prince to princess. Somewhere on the map, there is a direction that means, loosely, "the female version of."

Tap Show a direction in the map above to see the two parallel arrows. Other directions turn up the same way: a direction for "young version of" (dog toward puppy, cat toward kitten), a direction for country toward its capital city, a direction for a verb toward its past tense. None of these were programmed. They are patterns in how people write, and the map absorbed them.

Two honest caveats. First, these directions are approximate. They work beautifully on the textbook examples and get messier on ordinary words. Second, the map also absorbs patterns you would not want. If the text it learned from tends to pair certain jobs with one gender, the map will lean that way too. Researchers have spent years measuring and trying to reduce those leanings, and it remains an active problem.

Why the real map has thousands of directions

A flat map has two directions: across and up. That is enough to show the idea, and not nearly enough to hold meaning. Think about the word "bank." It should be near money words and near river words. Think about "cold." It should be near weather, near drinks, near illness, and near unfriendly people. On a flat page, you cannot put one dot near all of those places at once without everything piling into the middle.

So real embeddings use far more directions. Each direction is one more number in the list, and each extra number gives the map one more way to separate things that should be apart while keeping close the things that should be close. As of October 2026, OpenAI's smaller embedding model produces lists of 1,536 numbers by default, and its larger model and Google's Gemini embedding models produce 3,072.

You cannot picture a map with three thousand directions, and neither can the people who build them. That is fine. Everything you saw on the flat map still holds: nearby means related, directions carry meaning, and opposites that share a topic still sit close. The extra directions just give the map enough room to do the job without crowding.

One practical consequence: the numbers from one embedding model mean nothing to another. Each model draws its own map with its own directions. A tool that switches embedding models has to re-map every document it holds, the way you would have to redo every address if a city renamed all of its streets at once.

How embeddings power search and RAG

This is where embeddings stop being a curiosity and start running things you use every day.

Old-fashioned search matches words. If your question says "cancellation" and the document says "if you back out of a booking," a word match finds nothing. Embedding search, often called semantic search (search by meaning), works differently. The tool turns every passage in the library into an address on the map ahead of time. When you ask a question, it turns your question into an address too, then grabs the passages whose addresses sit closest. Shared meaning matters. Shared words do not.

If you took the RAG lesson, this is the "fingerprint of meaning" the librarian uses to find the right chunk. The librarian is walking the map. Every strength and every weakness from that lesson traces back here. Search can find a paragraph that uses none of your words, which is the strength. Search can also grab the deposit paragraph when you asked about cancellations, because those two live on the same block, which is the weakness.

The same idea shows up in places that do not look like search at all:

• Recommendations. "Customers who read this also read" can be as simple as "articles whose addresses sit near this one."
• Sorting and grouping. A pile of thousands of customer reviews can be grouped by what they are about, because reviews on the same topic land in the same neighborhood even when the wording differs.
• Spotting duplicates. Two support tickets that describe the same problem in different words end up at nearly the same address.
• Matching across languages. Some models place "thank you" and "gracias" near each other, so a question in one language can find an answer written in another.

Inside a chat model, the same kind of map is the very first step. Each token gets turned into a list of numbers before the model does anything else with it. Everything the later lessons in this module cover, including attention and sampling, happens to those lists, not to the letters you typed.

Try this yourself

You cannot see the numbers inside a chat app, but you can watch the habit they create. Open whichever AI chat you already use and paste this:

Here is a question and five sentences. Rank the sentences from most to least related to the question, and for each one, say whether it shares any actual words with the question.

Question: Can I get my money back if I change my mind?

1. Refunds are available within 30 days of purchase.
2. Returns are accepted with a receipt.
3. Our store hours change in the summer.
4. We accept cash, card, and checks.
5. If a customer has second thoughts, we will make it right.

Watch for two things. First, sentence 5 should rank high even though it shares almost no words with the question. That is meaning beating word matching. Second, sentence 3 shares the word "change" and still ranks near the bottom. Then follow up with this:

Give me five pairs of words that are opposites but show up in almost the same sentences, like hot and cold.

Every pair it gives you is a pair of next-door neighbors on the map, and a pair that a search by meaning could mix up. Keep that list in mind the next time an AI tool hands you a confident answer from the wrong paragraph.