🗂️ LlamaIndex 🦙

LlamaIndex (GPT Index) is a data framework for your LLM application. Building with LlamaIndex typically involves working with LlamaIndex core and a chosen set of integrations (or plugins). There are two ways to start building with LlamaIndex in Python:

1. Starter:llama-index.A starter Python package that includes core LlamaIndex as well as a selection of integrations.

2. Customized:llama-index-core.Install core LlamaIndex and add your chosen LlamaIndex integration packages onLlamaHub that are required for your application. There are over 300 LlamaIndex integration packages that work seamlessly with core, allowing you to build with your preferred LLM, embedding, and vector store providers.

The LlamaIndex Python library is namespaced such that import statements which includecoreimply that the core package is being used. In contrast, those statements withoutcoreimply that an integration package is being used.

# typical pattern
fromllama_index.core.xxximportClassABC# core submodule xxx
fromllama_index.xxx.yyyimport(
SubclassABC,
)# integration yyy for submodule xxx

# concrete example
fromllama_index.core.llmsimportLLM
fromllama_index.llms.openaiimportOpenAI

Important Links

LlamaIndex.TS(Typescript/Javascript)

Documentation

Twitter

Discord

Ecosystem

• LlamaHub(community library of data loaders)
• LlamaLab(cutting-edge AGI projects using LlamaIndex)

🚀 Overview

NOTE:This README is not updated as frequently as the documentation. Please check out the documentation above for the latest updates!

Context

• LLMs are a phenomenal piece of technology for knowledge generation and reasoning. They are pre-trained on large amounts of publicly available data.
• How do we best augment LLMs with our own private data?

We need a comprehensive toolkit to help perform this data augmentation for LLMs.

Proposed Solution

That's whereLlamaIndexcomes in. LlamaIndex is a "data framework" to help you build LLM apps. It provides the following tools:

• Offersdata connectorsto ingest your existing data sources and data formats (APIs, PDFs, docs, SQL, etc.).
• Provides ways tostructure your data(indices, graphs) so that this data can be easily used with LLMs.
• Provides anadvanced retrieval/query interface over your data:Feed in any LLM input prompt, get back retrieved context and knowledge-augmented output.
• Allows easy integrations with your outer application framework (e.g. with LangChain, Flask, Docker, ChatGPT, or anything else).

LlamaIndex provides tools for both beginner users and advanced users. Our high-level API allows beginner users to use LlamaIndex to ingest and query their data in 5 lines of code. Our lower-level APIs allow advanced users to customize and extend any module (data connectors, indices, retrievers, query engines, reranking modules), to fit their needs.

💡 Contributing

Interested in contributing? Contributions to LlamaIndex core as well as contributing integrations that build on the core are both accepted and highly encouraged! See ourContribution Guidefor more details.

📄 Documentation

Full documentation can be foundhere

Please check it out for the most up-to-date tutorials, how-to guides, references, and other resources!

💻 Example Usage

#custom selection of integrations to work with core
pip install llama-index-core
pip install llama-index-llms-openai
pip install llama-index-llms-replicate
pip install llama-index-embeddings-huggingface

Examples are in thedocs/examplesfolder. Indices are in theindicesfolder (see list of indices below).

To build a simple vector store index using OpenAI:

importos

os.environ["OPENAI_API_KEY"]="YOUR_OPENAI_API_KEY"

fromllama_index.coreimportVectorStoreIndex,SimpleDirectoryReader

documents=SimpleDirectoryReader("YOUR_DATA_DIRECTORY").load_data()
index=VectorStoreIndex.from_documents(documents)

To build a simple vector store index using non-OpenAI LLMs, e.g. Llama 2 hosted onReplicate,where you can easily create a free trial API token:

importos

os.environ["REPLICATE_API_TOKEN"]="YOUR_REPLICATE_API_TOKEN"

fromllama_index.coreimportSettings,VectorStoreIndex,SimpleDirectoryReader
fromllama_index.embeddings.huggingfaceimportHuggingFaceEmbedding
fromllama_index.llms.replicateimportReplicate
fromtransformersimportAutoTokenizer

# set the LLM
llama2_7b_chat="meta/llama-2-7b-chat:8e6975e5ed6174911a6ff3d60540dfd4844201974602551e10e9e87ab143d81e"
Settings.llm=Replicate(
model=llama2_7b_chat,
temperature=0.01,
additional_kwargs={"top_p":1,"max_new_tokens":300},
)

# set tokenizer to match LLM
Settings.tokenizer=AutoTokenizer.from_pretrained(
"NousResearch/Llama-2-7b-chat-hf"
)

# set the embed model
Settings.embed_model=HuggingFaceEmbedding(
model_name="BAAI/bge-small-en-v1.5"
)

documents=SimpleDirectoryReader("YOUR_DATA_DIRECTORY").load_data()
index=VectorStoreIndex.from_documents(
documents,
)

To query:

query_engine=index.as_query_engine()
query_engine.query("YOUR_QUESTION")

By default, data is stored in-memory. To persist to disk (under./storage):

index.storage_context.persist()

To reload from disk:

fromllama_index.coreimportStorageContext,load_index_from_storage

# rebuild storage context
storage_context=StorageContext.from_defaults(persist_dir="./storage")
# load index
index=load_index_from_storage(storage_context)

🔧 Dependencies

We use poetry as the package manager for all Python packages. As a result, the dependencies of each Python package can be found by referencing thepyproject.toml file in each of the package's folders.

cd<desired-package-folder>
pip install poetry
poetry install --with dev

📖 Citation

Reference to cite if you use LlamaIndex in a paper:

@software{Liu_LlamaIndex_2022,
author = {Liu, Jerry},
doi = {10.5281/zenodo.1234},
month = {11},
title = {{LlamaIndex}},
url = {https://github.com/jerryjliu/llama_index},
year = {2022}
}