Overview of LlamaIndex v0.10

I found the following article interesting, so I have summarized it briefly.

1. LlamaIndex v0.10

“LlamaIndex v0.10” is the largest update to date.
It marks a major step for “LlamaIndex” to become a next-generation, production-ready LLM application framework.

The main updates are as follows:

2. Created llama-index-core and separated integrations into individual packages

“LlamaIndex” has evolved into an extensive toolkit containing hundreds of “integrations”.

There is even more functionality included across “llama_index” and “llama-hub”. The rapid growth of the LlamaIndex ecosystem is remarkable, but it also comes with growing pains.

To address this, we created “llama-index-core” and separated the “integrations” into individual packages.

2-1. Creation of llama-index-core

We have created a slimmed-down package that contains the core LlamaIndex abstractions and components.

2-2. Separation of integrations into individual packages

All “integrations” are now available as individual packages. This includes all “integrations,” including those on “LlamaHub.” For a complete list of all packages, please refer to the Notion registry page.

2-3. llama-index

The “llama-index” package still exists and imports “llama-index-core” and a minimal set of integrations. Since it uses OpenAI by default, this includes the OpenAI package and SimpleDirectoryReader.

For those who do not wish to migrate to v0.10 and want to continue using the current “LlamaIndex,” we will provide “llama-index-legacy” (pinned to the latest release 0.9.48) for the time being.

2-4. Revamp of folder structure

The folder structure of the "llama_index" repository has been completely revamped.

The important folders are as follows:

The other folders are as follows:

The Integration and Pack subdirectories represent individual packages. The folder names correspond to the package names.

Within each package folder, source files are placed in the same path used for imports. This folder structure allows the top-level "llama_index" namespace to be maintained during imports.

2-5. Usage examples for Integrations

All third-party "Integrations" are now placed under "llama-index-integrations". There are 19 folders here.

The main Integration categories are as follows:

All other categories are as follows:

Usage examples for "Integrations" are as follows:

・Usage example for Anthropic LLM

pip install llama-index-llms-anthropic

from llama_index.llms.anthropic import Anthropic
llm = Anthropic(api_key="<api_key>")

・Usage example for data loaders

pip install llama-index-readers-notion

from llama_index.readers.notion import NotionPageReader
integration_token = os.getenv("NOTION_INTEGRATION_TOKEN")
page_ids = ["<page_id>"]
reader = NotionPageReader(integration_token=integration_token)
documents = reader.load_data(page_ids=page_ids)

・Usage example for LlamaPacks

pip install llama-index-packs-sentence-window-retriever

from llama_index.packs.sentence_window_retriever import SentenceWindowRetrieverPack
sentence_window_retriever_pack = SentenceWindowRetrieverPack(
  documents
)
response = sentence_window_retriever_pack.run("Tell me a bout a Music celebritiy.")

3. Managing all Integrations on LlamaHub

3-1. Expansion of LlamaHub

While the existing "LlamaHub" has provided "loaders," "tools," "packs," and "datasets," it will now provide all Integrations, including "LLMs," "embeddings," "vector stores," and "callbacks."

This effort is still a work in progress. We will update the site within a few weeks. In the meantime, please check the Notion package registry.

"LlamaHub" will continue to exist, but the "llama-hub" repository will be deprecated.

3-2. Obtaining Integrations from LlamaHub

The general UX for obtaining integrations through "LlamaHub" is download syntax such as download_loader and download_llama_pack.

4. ServiceContext is deprecated

We are deprecating "ServiceContext" to improve the developer experience of "LlamaIndex".

"ServiceContext" existed as a general configuration container containing LLMs, embedding models, callbacks, etc. It was created before proper LLM, embedding, and prompt abstractions were in place, and was intended to be an intermediate user-facing layer that allowed users to define these parameters.

However, over time, this object became difficult to use. Passing the entire service_context container to arbitrary modules made it difficult to infer which components were actually being used. Since all modules used OpenAI by default, users were unnecessarily prompted to provide an OpenAI key even if they wanted to use a local model. It was also difficult to import and type.

Another related issue was that if you had custom models, especially custom callbacks, you had to manually pass the service_context to every module. This was a tedious task and easy for users to forget, leading to missed callbacks or inconsistent model usage.

To address this, we have made the following changes:

All references to "ServiceContext" in the documentation have been removed and changed to use either direct modules or global configuration objects.

Examples of usage replacing ServiceContext are as follows:

・Pass LLM and embedding models directly

from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.llms.openai import OpenAI
from llama_index.core.callbacks import CallbackManager

embed_model = OpenAIEmbedding()
llm = OpenAI()
callback_manager = CallbackManager()
index = VectorStoreIndex.from_documents(
 documents, embed_model=embed_model, callback_manager=callback_manager
)
query_engine = index.as_query_engine(llm=llm)

・Define global settings

from llama_index.core.settings import Settings
Settings.llm = llm
Settings.embed_model = embed_model
Settings.callback_manager = callback_manager
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()

5. Migrating to v0.10

Migrating to "LlamaIndex v0.10" mainly requires the following two tasks:

We also include a tool to automatically upgrade existing code to v0.10 and provide a comprehensive migration guide.

llamaindex-cli upgrade <source-dir>