What is the difference between an embedding and a raw feature vector?

Raw feature vectors are often manually engineered or simple counts (like word frequency), while embeddings are learned representations from a neural network trained to capture semantic relationships, generally yielding much better similarity behavior.

How are embeddings compared to each other?

Typically using distance or similarity metrics such as cosine similarity or Euclidean distance; vectors that are close together are considered semantically similar.

Can I mix embeddings from different models?

No, generally not. Embeddings from different models live in different, incompatible vector spaces, so a system must use one consistent embedding model for both storing and querying data.

What size are embedding vectors?

It varies by model, commonly ranging from a few hundred to a few thousand dimensions; higher dimensionality can capture more nuance but costs more storage and compute.

Are embeddings only for text?

No. Embeddings can represent images, audio, video, code, and even multimodal combinations, as long as a model has been trained to map that data type into a shared vector space.

AI Fundamentals

Embeddings

IntermediateConcept2.1K learners

#Embeddings #AIFundamentals#Concept#Intermediate#VectorSearch #RAG #NeuralNetwork #MachineLearning #ArtificialIntelligence#Glossary#SkillVeris

Definition

Embeddings are dense numerical vector representations of data — such as words, sentences, images, or documents — learned so that semantically similar items are positioned close together in vector space. Produced by a neural network trained on large datasets, embeddings let machines compare, search, and reason about meaning mathematically, using distance or similarity metrics between vectors.

Overview

Raw data like text or images is not directly usable by most machine learning algorithms in its native form. Embeddings solve this by mapping such data into fixed-length numerical vectors, typically ranging from tens to thousands of dimensions, where the geometric relationships between vectors capture meaningful semantic relationships. For example, in a well-trained word embedding space, vectors for 'king' and 'queen' are close together, and vector arithmetic can even capture analogies like king minus man plus woman approximating queen. Embeddings are learned rather than manually specified, typically as a byproduct of training a neural network on a task like predicting missing words (in early models like word2vec) or as an output layer of a large language model or dedicated embedding model. Sentence and document embeddings extend the idea to larger units of text, enabling comparison of entire passages for similarity. The practical value of embeddings lies in enabling similarity-based operations that would otherwise require exact keyword matching: semantic search finds documents with related meaning even if they don't share exact words, clustering groups similar items automatically, and recommendation systems find related products or content. Embeddings are the foundational technology behind vector search and retrieval-augmented generation (RAG), where a query is embedded and compared against embedded document chunks to find relevant context for a language model. Different embedding models trade off dimensionality, quality, domain specialization (e.g., code embeddings vs. general text embeddings), and computational cost. Because embeddings from different models are generally not directly comparable, a RAG or search system must use the same embedding model consistently for both indexing and querying.

Key Concepts

Represent data as dense numerical vectors capturing semantic meaning
Semantically similar items are positioned close together in vector space
Learned by neural networks trained on large text, image, or multimodal datasets
Enable similarity search using distance metrics like cosine similarity or dot product
Available at multiple granularities: word, sentence, document, or image embeddings
Form the foundation of vector search and retrieval-augmented generation (RAG)
Must be generated by the same model for indexing and querying to be comparable
Dimensionality and quality vary by model, trading off accuracy, cost, and speed

Use Cases

Semantic search over documents, code, or product catalogs

Retrieval-augmented generation (RAG) context retrieval

Recommendation systems based on content or user similarity

Duplicate and near-duplicate detection

Clustering and topic modeling of large text corpora

Image similarity search and reverse image lookup

Anomaly detection based on distance from typical embeddings

Multilingual search by mapping different languages into a shared embedding space

Frequently Asked Questions

From the Blog

AI & Technology

RAG Explained: Retrieval-Augmented Generation

RAG is how you give an LLM access to your own private data without training a new model. This guide explains the full pipeline — chunking, embeddings, vector search, and augmented generation — with a working Python example using open-source tools.

How Large Language Models Actually Work

LLMs seem magical until you understand what they are: next-token predictors trained on massive text corpora. This guide explains tokenisation, embeddings, the transformer architecture, attention mechanism, and how training works — without requiring a maths degree.