Switching LlamaIndex's embedding model to Multilingual-E5-large

I have summarized the steps to switch the embedding model of LlamaIndex to Multilingual-E5-large.

1. Multilingual-E5-large

Multilingual-E5-large is a model for multilingual text embedding. The sequence length is 514, and the embedding dimension is 1024.

By switching from the default text-embedding-ada-002, you can expect to eliminate embedding API usage costs and improve performance.

2. Preparing the document

First, prepare a document containing the specialized knowledge you want to teach the chatbot.

This time, I prepared a document with the synopsis of "Bocchi the Rock!" from Mangapedia.

・bocchi.txt

3. Execution on Colab

The steps for execution on Google Colab are as follows.

(1) Install the packages.

# パッケージのインストール
!pip install llama-index
!pip install sentence_transformers

(2) Prepare environment variables.
In the code below, specify your <OpenAI_API_token> for the OpenAI API. (Paid)

import os
os.environ["OPENAI_API_KEY"] = "<OpenAI_APIのトークン>"

(3) Set the log level.

import logging
import sys

# ログレベルの設定
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG, force=True)

(4) Create a data folder in Colab and place the document.
Display the file list using the folder icon on the far left, right-click to create a new folder named 'data', and drag and drop the document.

(5) Load the document.

from llama_index import SimpleDirectoryReader

# ドキュメントの読み込み
documents = SimpleDirectoryReader("data").load_data()
print("documents :", documents)

(6) Create the index.
Set the embedding model to "intfloat/multilingual-e5-large".

from langchain.embeddings import HuggingFaceEmbeddings
from llama_index import GPTVectorStoreIndex, ServiceContext, LangchainEmbedding

# 埋め込みモデルの準備
embed_model = LangchainEmbedding(HuggingFaceEmbeddings(
    model_name="intfloat/multilingual-e5-large"
))

# ServiceContextの準備
service_context = ServiceContext.from_defaults(
    embed_model=embed_model
)

# インデックスの生成
index = GPTVectorStoreIndex.from_documents(
    documents, # ドキュメント
    service_context=service_context, # ServiceContext
)

(7) Create the query engine.
Specify "3" for the number of chunks to retrieve as context.

# クエリエンジンの作成
query_engine = index.as_query_engine(
    similarity_top_k=3  # 取得するチャンク数 (default:2)
)

(8) Question answering.
I attempted to specify three chunks as context and successfully derived the correct answer.

# 質問応答
response = query_engine.query("ぼっちちゃんの髪の色は？")
print(response)

DEBUG:llama_index.indices.utils:> Top 3 nodes:
> [Node f1c8d99c-eba5-489e-8d5a-0e0659798f3b] [Similarity score:             0.793081] 秀華高校に通う女子。桃色の髪を無造作に伸ばし、いつもジャージを身につけている。自他共に認める引きこもり一歩手前の「陰キャ」で、承認欲求が人一倍強いにもかかわらず、臆病な性格で人と接するのを極度に...
> [Node 48757bed-c249-44d4-b6ce-a0610b9a9bf0] [Similarity score:             0.791189] ばし放題で前髪で目をつねに隠している。それに加えて野暮ったいジャージ姿でいるため気づかれていないが、実は同性すら見とれるほどの美少女。黙って着飾っていれば「アイドル事務所に入れる」「ビジュアル担...
> [Node 1245dda8-1f92-4ca9-a60c-563c4f3b0a29] [Similarity score:             0.785498] 下北沢高校に通う女子。後藤ひとりより1学年上。ライトイエロー色の髪をサイドテールにセットし、派手目のファッションを好んで着ている。幼い頃に母親と死別しており、父親も多忙なためさびしい日々を送って...
DDEBUG:openai:message='Request to OpenAI API' method=post path=https://api.openai.com/v1/completions
DEBUG:openai:api_version=None data='{"prompt": "Context information is below.\\n---------------------\\n\\u79c0\\u83ef\\u9ad8\\u6821\\u306b\\...", "stream": false, "model": "text-davinci-003", "temperature": 0.0, "max_tokens": 2785}' message='Post details'
DEBUG:urllib3.connectionpool:https://api.openai.com:443 "POST /v1/completions HTTP/1.1" 200 None
DEBUG:openai:message='OpenAI API response' path=https://api.openai.com/v1/completions processing_ms=1110 request_id=32c98637be561ef4b8a98d585c5d656e response_code=200
DEBUG:llama_index.llm_predictor.base:
桃色

桃色

Related