Vector Databases and Semantic Search: Finding Information by Meaning

Reading objectives and introduction

This is an English translation of the published reading. We will learn about semantic gaps, vector embeddings, and vector databases, understand how semantic search works, observe applications at the individual and organizational levels, and examine its relationship with RAG.

Imagining someone asking about "the customer experience strategy discussed last month." You remember the concept, but searching with those words returns 847 irrelevant results, since the actual document is titled "Customer Journey Optimization." This is an example of original material design: people remember concepts, while the words appearing in the document may differ.

People might say "email with activity ideas" or "photo of a beach sunset," while the system often relies on name, date, type, and tags. The gap between how people understand meaning and how data is organized is known as the semantic gap.

1. From token to comparable representations

Text can be first tokenized into tokens, then passed into the model's numerical computation. For example, "The quick brown fox" is illustrated as being segmented into The, quick, brown, fox; in practice, tokenizers do not always split on whole words.

Embedding represents words, sentences, or fragments as vectors, i.e., a set of numbers. The original figure assumes a 512-dimensional representation: "quick" corresponds to [0.2, -0.5, 0.8, 0.1, -0.3, …] , fast corresponds to [0.3, -0.4, 0.7, 0.2, -0.2, …] In an appropriate shared representation space, semantically similar items may exhibit high similarity.

Note: These values are illustrative and the five dimensions shown do not prove that the vectors are sufficiently similar. The representation of tokens within language models is not necessarily the same as sentence embeddings used in document retrieval; computing similarity during vector retrieval does not imply that the model is relearning.

Therefore, searching for "quick vehicles" might yield content written as "fast cars," without requiring exact lexical match.

II. Keyword Search and Semantic Search

The original material summarizes traditional search using "exact matching," but actual keyword systems can also include synonym matching, word segmentation, and ranking mechanisms. The two approaches can be combined; there's no need to choose between exact matching and semantic understanding.

Three: Why is embedding like a "meaning fingerprint"?

The original explanation uses fingerprints as an analogy: content corresponds to digital representations, and similar content can be compared. Sunset in a mountainous area and sunset by the beach occur at different locations, but share concepts such as sunset and warm colors, thus possibly being close in an appropriate model.

It also uses "elevation" and "warmth level" to explain vector dimensions. This is a simplification for building intuition, as individual dimensions in the true model are typically not directly named as clear attributes.

Vectors are not the only or lossless form of identity verification and cannot be used to replace the original text, hash validation, or factual verification. On the contrary, similar meanings may share many semantic features, for example, "refundable" and "non-refundable".

View textual explanation of the diagram

Vector similarity helps identify candidate materials that are semantically related. The proximity between "refundable" and "non-refundable" in the figure indicates that semantic similarity does not guarantee consistent conclusions. Further review of conditions, versions, and the original text is still required; actual embeddings typically have more dimensions.

When looking at diagrams, note that proximity indicates relevance, not that two sentences can be interchanged. After retrieving content, you must carefully examine negations, conditions, and applicable scopes.

IV. How to query a vector database

The process typically involves: extracting and organizing content; forming representations using embedded models; storing vectors along with necessary metadata; converting queries into comparable representations; identifying similar items, and then returning the corresponding original documents.

To speed up queries over large datasets, the system uses methods such as indexing and approximate search. This can be analogized to organizing data based on content relationships, rather than simply alphabetically or by filename.

Vector databases can support long-term document retrieval for Agents, but not all memories need to be stored as vectors. Conversation states, precise business records, permissions, and task progress may instead be managed by traditional databases or other structured systems. Storing data as vectors does not automatically mean the system has learned the business domain or has become an expert.

Five. Examples in Individuals and Organizations

Organize materials discovery

The original material, as explained by a consulting firm: reports, case studies, and personnel expertise are scattered across different departments with inconsistent terminology, leading to redundant creation. Semantic search enables users to find relevant materials and experiences using natural language, facilitating knowledge sharing.

The statement "knowledge reuse increased by 50%" lacks verifiable sources and measurement criteria, and thus is presented only as an anonymous teaching case and not as a generalizable result applicable to all organizations.

Customer Service Knowledge Base

"Computer won't power on" "Boot failure" "Power supply issue" may be related, but are not always the same fault. Semantic retrieval can provide potential troubleshooting steps, helping service personnel quickly identify the right knowledge.

The original case study claims a 40% reduction in average handling time and improved first-time resolution rate. Without original evidence, it cannot be used as a basis for own expectations; it is also necessary to verify that steps applicable to the specific device and version have been identified.

Meeting notes and client preparation

The original document describes a market representative who needs to locate retail customer challenges from disorganized notes prior to a call. Natural language queries may cross different meetings to retrieve relevant content, reducing the risk of missing information due to variations in wording.

The transition from 20 minutes to a few seconds in the text is an example, not a verified statistic. It suggests that preparation time, omissions, and redundant work can be measured, not just the number of search results.

Professional document research

Legal concepts may be expressed in various ways, and semantic retrieval can help identify relevant cases. The original text treats "duty of loyalty" and "duty of care" as related expressions, but they are not simply equivalent in law; further professional review of the specific concepts, legal jurisdiction, and court rulings is still required.

The original case "reduced research time by 60%" also lacks a source. Broadening the search scope might be beneficial, or it might yield more irrelevant results.

Other purposes

You can search notes by idea, images by concept, and files by content rather than name; it can also be used for customer service knowledge, onboarding policies, and cross-terminology research. Actual evaluation should verify that the retrieved materials are accurate, complete, up-to-date, and accessible.

Six: How do images and sounds enter vector space?

image

A 256 × 256 image has 65,536 pixels, and an RGB image has three channel values per pixel. The model can learn edges, textures, and more complex structures from pixels or image blocks, ultimately producing representations in hundreds or thousands of dimensions.

The original states that images lack word-level tokenization, which helps distinguish media; but some visual architectures use image blocks or discrete tokens, and cannot be generalized to claim that image processing never involves tokens.

Sound

The original explanation outlines a common pathway: waveform represents a signal varying over time; after conversion to a spectrogram, one axis is time and the other is frequency, with intensity reflecting the corresponding energy; the network then forms vectors from low-level bands and rhythms, up to phonetic units or more abstract features.

This is not the only path for all audio models, as some directly process waveforms or use alternative representations. Audio embeddings might focus on speech content, speaker identity, or acoustic events and may not automatically capture all meanings and emotions.

Seven: Why does cross-modal retrieval require alignment?

Text, images, audio, and video can all be represented as vectors, but it's not valid to directly compare them simply because they are all composed of numbers. Cross-modal models need to train or map outputs from different encoders into a mutually compatible space.

The original material uses examples of "golden retriever puppies" in text, images, and vocalizations: within a system aligning concepts and media, relevant assets can be found across different formats. Audio of a dog's bark alone is not reliably sufficient to identify the breed; this only indicates a possible retrieval relationship.

View textual explanation of the diagram

Images and texts are separately encoded and paired during training to bring related image-text pairs close in a comparison space. For audio to be searched by text, corresponding audio-text alignment is required. Even if different models produce outputs with the same dimensional output, the numerical values do not necessarily have the same meaning; the results still require verification.

In the figure, each modality first passes through an appropriate encoder before entering a comparable space; arbitrarily mixing vectors from different models cannot achieve reliable cross-modal search.

Applications may include: searching for cloud network diagrams using text, identifying "vibrant" video clips through visual and auditory cues, or locating articles, images, and related explanations around historical sites. Different sources still require licensing, quality, and origin verification.

Eight: How is it connected to RAG?

Improves the way information is discovered from data, offering potential to reduce query times, uncover unexpected associations, and assist in leveraging organizational knowledge. It does not guarantee "understanding everything."

After retrieving the relevant materials, you can hand them over to a generative model to organize the response, and then proceed to the next section. RAG and business knowledge . Tokenization, representation, indexing, retrieval, and generation are distinct stages; do not treat them as a single, automated process.

Video script: From unable to find files to discovering by meaning

A familiar challenge is that when searching for a solution related to customer experience from last month using the term "customer experience solution," 847 irrelevant results appear. The actual document might be titled "customer journey optimization." This isn't a lack of information; it's simply a mismatch between the concept remembered and the actual terms used in the documents.

People might think of things like "email for activity idea," "summer trip photos," or "document for discussing a new strategy," but the system often relies on exact names, keywords, and text. This gap is known as the semantic gap. Keyword matching doesn't necessarily recognize that quick and fast are similar in meaning, nor does it understand that customer experience and customer journey are related.

Vector embeddings provide a way to connect things. Think of them as meaning fingerprints: words, sentences, or documents are represented as a set of numbers, and the similarity relationships learned by the model can help identify content that differs in wording.

The original explanation said that quick and fast are quite similar, while slow might differ. Note that opposite or related terms might also be close in certain contexts, so one cannot directly equate proximity with logical equivalence. Fingerprints are only analogies and do not guarantee uniqueness or losslessness.

Text, images, and sound can all be represented numerically, but the emphasis of representation differs; cross-media comparison is only possible after appropriate training and alignment, and not every numerical vector is inherently interoperable.

Vector databases store these representations and associated information. Documents are first computed into embeddings and indexed; during search, the query is also converted into a compatible representation, and similar items are then located and returned. It functions like a document curator that helps understand the direction of the query, rather than merely matching against original words.

These mechanisms can help the Agent retrieve context, conversation, or organizational knowledge, enabling the system to access necessary information for the current task. However, vector databases are not equivalent to complete memory, and they do not mean the model automatically learns from every conversation or becomes an expert in a particular domain.

Extending from surface-level matching to meaning discovery can enhance the research experience and reduce missed detections due to different naming conventions. A reliable implementation still requires verification of relevance, data quality, access permissions, and currency, rather than relying solely on the notion of "AI understands me."

Related Reading

After retrieving the materials, you can proceed by RAG Bring in the response; how does indexing maintain reliability, need to return to Data pipelines Check quality and update.

Reference and Extended Reading

The following is an editorial addition and does not constitute endorsement of the anonymous efficiency figures.

• Google: Embeddings : Chinese, query vector representation and similarity relationships.
• Faiss Official Documentation : English, understand vector similarity search and indexing.
• Pinecone: Vector Embeddings : English, investigate the relationship among embedding, indexing, and application.