Are embeddings made of tokens?

Embeddings are computed from tokens. Text is tokenized first, each token is turned into a vector, and the model combines those vectors. Embedding models then pool them into one vector for the whole input, which represents its meaning rather than its individual tokens.

Do embeddings cost tokens?

Creating embeddings through an API is usually billed per input token, but it is typically much cheaper than generation and done once per chunk. After that, stored embeddings can be searched many times without paying generation costs again. Local embedding models remove the API cost entirely.

Why do RAG systems use embeddings instead of keywords?

Embeddings match meaning, so a query can find relevant text even when wording differs. Many production systems combine embeddings with keyword search, because keywords still help with exact names, codes, and identifiers that embeddings can blur. This combination is usually called hybrid search.

Embeddings vs Tokens

Tokens are the pieces of text, usually word fragments, that a tokenizer turns into integer IDs for a model to read and that providers bill by. Embeddings are dense numeric vectors that represent meaning, so similar texts sit close together. Models turn tokens into embeddings internally, and RAG systems use embeddings to find relevant text before spending tokens.

Tokens, briefly

A tokenizer splits text into pieces from a fixed vocabulary, often subwords produced by byte-pair encoding or similar methods. Common words may be one token; rare words split into several. Each piece maps to an integer ID. Models read sequences of IDs, context windows are measured in tokens, and API pricing is per input and output token. The same text can produce different token counts on different models because tokenizers differ.

What an embedding is

An embedding is a list of numbers, often hundreds or thousands of dimensions, that places text in a vector space where similar meanings are near each other. Embedding models produce one vector for a sentence, paragraph, or chunk. Comparing two vectors with cosine similarity or a dot product gives a measure of semantic similarity, even when the texts share no words.

How they connect inside a model

The first layer of a language model is an embedding table: each token ID is replaced by a learned vector. Those token embeddings pass through the network and become contextual representations. Embedding models used for retrieval take this further and pool the representations into a single vector for the whole input. So tokens are the input format, and embeddings are the representation the model actually computes with.

Why the difference matters for cost

You pay for tokens. Every token sent or generated costs money and time. Embeddings are usually cheap to compute once and store. A retrieval system embeds documents ahead of time, embeds each query, and finds the closest chunks. Sending a few retrieved chunks costs far fewer tokens than sending whole documents or regenerating answers from scratch.

Common confusion

Token counts do not measure meaning, and embeddings do not measure length. Two texts with similar token counts can mean very different things; two very different lengths can be close in embedding space. Embedding models also have their own token limits, so long documents must be chunked before embedding.

Where RDK fits

RDK indexes vaults and codebases as chunks with embeddings, encrypted as private chunks on the RDK network. Agents search those embeddings before calling an LLM, so known answers come back as retrieved text instead of new tokens. That is how retrieval turns embedding work, done once, into token savings on every repeated query.

Counting tokens before you send

Use the tokenizer for your target model to count tokens before a call, so you can trim context to budget. For retrieval, count the tokens in the chunks you plan to send, not the documents they came from. Chunk sizes for embedding should also respect the embedding model's own token limit, or text beyond it is ignored.

Choosing an embedding model

Embedding models differ in dimensions, input limit, language coverage, and domain fit. Larger vectors can capture more nuance but cost more to store and search. Test candidates on your own queries and documents, and keep the same model for indexing and querying, because vectors from different models are not comparable.

Aspect	Tokens	Embeddings
What it is	Integer ID for a text piece	Vector of numbers encoding meaning
Produced by	A tokenizer	An embedding model or layer
Used for	Model input, context limits, billing	Similarity search, clustering, retrieval
Measures	Length	Meaning
Cost driver	Per token sent and generated	Computed once per chunk, then stored

Frequently asked questions

Are embeddings made of tokens?: Embeddings are computed from tokens. Text is tokenized first, each token is turned into a vector, and the model combines those vectors. Embedding models then pool them into one vector for the whole input, which represents its meaning rather than its individual tokens.
Do embeddings cost tokens?: Creating embeddings through an API is usually billed per input token, but it is typically much cheaper than generation and done once per chunk. After that, stored embeddings can be searched many times without paying generation costs again. Local embedding models remove the API cost entirely.
Why do RAG systems use embeddings instead of keywords?: Embeddings match meaning, so a query can find relevant text even when wording differs. Many production systems combine embeddings with keyword search, because keywords still help with exact names, codes, and identifiers that embeddings can blur. This combination is usually called hybrid search.