Trying out Question Answering with HyDE in LangChain

I have summarized my experience trying out question answering using "HyDE" with "LangChain".

Previous post

1. HyDE

"HyDE" (Hypothetical Document Embeddings) is a technique for performing document retrieval for question answering with higher accuracy.
In general question answering, questions are embedded to search for documents, but with "HyDE", a hypothetical response is generated without reading the documents, and that response is then embedded to search for the documents.

2. Generating embeddings with HyDE

The steps for generating embeddings with HyDE in Google Colab are as follows.

(1) Install the packages.

# パッケージのインストール
!pip install langchain
!pip install openai

(2) Prepare environment variables.
In the code below, specify your <OpenAI_API_token> with your OpenAI API token. (Paid service)

import os
os.environ["OPENAI_API_KEY"] = "<OpenAI_APIのトークン>"

(3) Prepare HypotheticalDocumentEmbedder.
The arguments are the LLM, the embedding model, and the prompt key.

from langchain.llms import OpenAI
from langchain.embeddings import OpenAIEmbeddings, HypotheticalDocumentEmbedder
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate

# HypotheticalDocumentEmbedderの準備
embeddings = HypotheticalDocumentEmbedder.from_llm(
    llm=OpenAI(), 
    base_embeddings=OpenAIEmbeddings(), 
    prompt_key="web_search"
)

HyDE comes with several prompts for generating hypothetical responses.

The web_search prompt is as follows.

web_search_template = """Please write a passage to answer the question 
Question: {QUESTION}
Passage:"""

(4) Generate embeddings from text.

# テキストから埋め込みを生成
result = embeddings.embed_query("Where is the Taj Mahal?")
print(len(result))
print(result)

1536
[-0.00999956764280796, 0.002605215646326542,　...]

3. Generating embeddings using multiple hypothetical responses

It is also possible to generate multiple hypothetical responses and combine their embeddings. By default, they are combined by taking the average.

(1) Increase the n and best_of arguments for OpenAI().

# HypotheticalDocumentEmbedderの準備
embeddings = HypotheticalDocumentEmbedder.from_llm(
    llm=OpenAI(n=4, best_of=4), 
    base_embeddings=OpenAIEmbeddings(), 
    prompt_key="web_search"
)

(2) Generate embeddings from text.

# テキストから埋め込みを生成
result = embeddings.embed_query("Where is the Taj Mahal?")
print(len(result))
print(result)

1536
[-0.011480985092930496, -0.004797816043719649, ... ]

4. Generating hypothetical responses with custom prompts

You can also generate hypothetical responses using custom prompts.
In this instance, I will try creating a Japanese prompt.

(1) Prepare the custom prompt.

# カスタムプロンプトの準備
prompt_template = """質問に回答する文章を書いてください
質問: {question}
回答:"""
prompt = PromptTemplate(input_variables=["question"], template=prompt_template)
llm_chain = LLMChain(llm=OpenAI(), prompt=prompt)

(2) Preparing the HypotheticalDocumentEmbedder with a custom prompt.
Configure it via LLMChain.

# カスタムプロンプトのHypotheticalDocumentEmbedderの準備
embeddings = HypotheticalDocumentEmbedder(
    llm_chain=llm_chain, 
    base_embeddings=OpenAIEmbeddings(), 
)

(3) Generate embeddings from text.

# テキストから埋め込みを生成
result = embeddings.embed_query("タージ・マハルはどこにありますか?")
print(len(result))
print(result)

1536
[-0.016797535121440887, 0.00605577789247036, ...]

5. Question Answering

The steps for question answering are as follows.

(1) Prepare the documents.
This time, I prepared documents using information about "Bocchi the Rock!" from Mangapedia.

・bocchi.txt

(2) Upload to Colab and load the documents.
Since it is in Japanese, I have set the separator to "。".

from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import FAISS

# ドキュメントの読み込み
with open("bocchi.txt") as f:
    bocchi_txt = f.read()
text_splitter = CharacterTextSplitter(
    chunk_size=500, 
    chunk_overlap=0, 
    separator="。"
)
texts = text_splitter.split_text(bocchi_txt)
print(len(texts))
print(texts)

22
['結束バンド\n後藤ひとりは友達を作れない陰キャでいつも一人で過ごしていたが、...]

(3) Search for similar sentences.
For the question "What is Nijika-chan's specialty instrument?", Nijika-chan's profile was successfully retrieved.

query = "虹花ちゃんの得意な楽器は?"

# 類似文章の検索
docsearch = FAISS.from_texts(texts, embeddings)
docs = docsearch.similarity_search(query, k=1)
print(docs[0].page_content)

...

伊地知虹夏(いじちにじか)
下北沢高校に通う女子。後藤ひとりより1学年上。...

(5) Execute question answering.
The answer is "drums", so it is correct.

# 質問応答の実行
from langchain.chains.question_answering import load_qa_chain
chain = load_qa_chain(OpenAI(), chain_type="stuff")
chain({"input_documents": docs, "question": query}, return_only_outputs=True)

{'output_text': ' ドラム'}

Next time