Summary of RAG Workflows

Vector Store Workflow for Conversational Exchanges:

Generate semantic embedding for each new conversation, add the message body to a vector store for retrieval, query the vector store for relevant messages to fill in the LLM context.

Modified Workflow for an Arbitrary Document:

Objectives

Deep learning models can handle semantics by vectorizing natural language.

Attention: Measure the importance of the input sequence.

When reading long articles, your focus naturally shifts from one word to another, depending on the context.

The mechanism mimic this behavior, allowing models to selectively concentrate on specific elements of the input data while ignoring others.

---

In this note, I'll explain how to use Langchain and other tools to embedding a document and why should we do this.