questions|feature requests

🤔 why autollm?

Simplify. Unify. Amplify.

Feature	AutoLLM	LangChain	LlamaIndex	LiteLLM
100+ LLMs	✅	✅	✅	✅
Unified API	✅	❌	❌	✅
20+ Vector Databases	✅	✅	✅	❌
Cost Calculation (100+ LLMs)	✅	❌	❌	✅
1-Line RAG LLM Engine	✅	❌	❌	❌
1-Line FastAPI	✅	❌	❌	❌

---

📦 installation

easily installautollmpackage with pip inPython>=3.8environment.

pip install autollm

for built-in data readers (github, pdf, docx, ipynb, epub, mbox, websites..), install with:

pip install autollm[readers]

---

🎯 quickstart

tutorials

• video tutorials:

  • Make RAG with API Just in a few Seconds!!!
  • Autollm automagically create llm apps in seconds
  • AutoLLM: Create RAG Based LLM Web Apps in SECONDS!
  • AutoLLM: Ship RAG based LLM Apps and API in Seconds
  • 🚀 AutoLLM: Unlock the Power of 100+ Language Models! Step-by-Step Tutorial

• blog posts:

  • Introduction to AutoLLM

• colab notebooks:

  • quickstart:

create a query engine in seconds

>>>fromautollmimportAutoQueryEngine,read_files_as_documents

>>>documents=read_files_as_documents(input_dir="path/to/documents")
>>>query_engine=AutoQueryEngine.from_defaults(documents)

>>>response=query_engine.query(
..."Why did SafeVideo AI develop this project?"
... )

>>>response.response
"Because they wanted to deploy rag based llm apis in no time!"

👉 advanced usage

>>>fromautollmimportAutoQueryEngine

>>>query_engine=AutoQueryEngine.from_defaults(
...documents=documents,
...llm_model="gpt-3.5-turbo",
...llm_max_tokens="256",
...llm_temperature="0.1",
...system_prompt='...',
...query_wrapper_prompt='...',
...enable_cost_calculator=True,
...embed_model="huggingface/BAAI/bge-large-zh",
...chunk_size=512,
...chunk_overlap=64,
...context_window=4096,
...similarity_top_k=3,
...response_mode="compact",
...structured_answer_filtering=False,
...vector_store_type="LanceDBVectorStore",
...lancedb_uri="./lancedb",
...lancedb_table_name="vectors",
...exist_ok=True,
...overwrite_existing=False,
... )

>>>response=query_engine.query("Who is SafeVideo AI?")

>>>print(response.response)
"A startup that provides self hosted AI API's for companies!"

convert it to a FastAPI app in 1-line

>>>importuvicorn

>>>fromautollmimportAutoFastAPI

>>>app=AutoFastAPI.from_query_engine(query_engine)

>>>uvicorn.run(app,host="0.0.0.0",port=8000)
INFO:Startedserverprocess[12345]
INFO:Waitingforapplicationstartup.
INFO:Applicationstartupcomplete.
INFO:Uvicornrunningonhttp://http://0.0.0.0:8000/

👉 advanced usage

>>>fromautollmimportAutoFastAPI

>>>app=AutoFastAPI.from_query_engine(
...query_engine,
...api_title='...',
...api_description='...',
...api_version='...',
...api_term_of_service='...',
)

>>>uvicorn.run(app,host="0.0.0.0",port=8000)
INFO:Startedserverprocess[12345]
INFO:Waitingforapplicationstartup.
INFO:Applicationstartupcomplete.
INFO:Uvicornrunningonhttp://http://0.0.0.0:8000/

---

🌟 features

supports100+ LLMs

>>>fromautollmimportAutoQueryEngine

>>>os.environ["HUGGINGFACE_API_KEY"]="huggingface_api_key"

>>>llm_model="huggingface/WizardLM/WizardCoder-Python-34B-V1.0"
>>>llm_api_base="https://my-endpoint.huggingface.cloud"

>>>AutoQueryEngine.from_defaults(
...documents='...',
...llm_model=llm_model,
...llm_api_base=llm_api_base,
... )

👉 more llms:

• huggingface - ollama example:

  >>>fromautollmimportAutoQueryEngine
  
  >>>llm_model="ollama/llama2"
  >>>llm_api_base="http://localhost:11434"
  
  >>>AutoQueryEngine.from_defaults(
  ...documents='...',
  ...llm_model=llm_model,
  ...llm_api_base=llm_api_base,
  ... )

• microsoft azure - openai example:

  >>>fromautollmimportAutoQueryEngine
  
  >>>os.environ["AZURE_API_KEY"]=""
  >>>os.environ["AZURE_API_BASE"]=""
  >>>os.environ["AZURE_API_VERSION"]=""
  
  >>>llm_model="azure/<your_deployment_name>")
  
  >>>AutoQueryEngine.from_defaults(
  ...documents='...',
  ...llm_model=llm_model
  ... )

• google - vertexai example:

  >>>fromautollmimportAutoQueryEngine
  
  >>>os.environ["VERTEXAI_PROJECT"]="hardy-device-38811"# Your Project ID`
  >>>os.environ["VERTEXAI_LOCATION"]="us-central1"# Your Location
  
  >>>llm_model="text-bison@001"
  
  >>>AutoQueryEngine.from_defaults(
  ...documents='...',
  ...llm_model=llm_model
  ... )

• aws bedrock - claude v2 example:

  >>>fromautollmimportAutoQueryEngine
  
  >>>os.environ["AWS_ACCESS_KEY_ID"]=""
  >>>os.environ["AWS_SECRET_ACCESS_KEY"]=""
  >>>os.environ["AWS_REGION_NAME"]=""
  
  >>>llm_model="anthropic.claude-v2"
  
  >>>AutoQueryEngine.from_defaults(
  ...documents='...',
  ...llm_model=llm_model
  ... )

supports20+ VectorDBs

🌟Pro Tip:autollmdefaults tolancedbas the vector store: it's setup-free, serverless, and 100x more cost-effective!

👉 more vectordbs:

• QdrantVectorStore example:

  >>>fromautollmimportAutoQueryEngine
  >>>importqdrant_client
  
  >>>vector_store_type="QdrantVectorStore"
  >>>client=qdrant_client.QdrantClient(
  ...url="http://<host>:<port>",
  ...api_key="<qdrant-api-key>"
  ... )
  >>>collection_name="quickstart"
  
  >>>AutoQueryEngine.from_defaults(
  ...documents='...',
  ...vector_store_type=vector_store_type,
  ...client=client,
  ...collection_name=collection_name,
  ... )

automated cost calculation for100+ LLMs

>>>fromautollmimportAutoServiceContext

>>>service_context=AutoServiceContext(enable_cost_calculation=True)

# Example verbose output after query
EmbeddingTokenUsage:7
LLMPromptTokenUsage:1482
LLMCompletionTokenUsage:47
LLMTotalTokenCost:$0.002317

create FastAPI App in 1-Line

👉 example

>>>fromautollmimportAutoFastAPI

>>>app=AutoFastAPI.from_config(config_path,env_path)

Here,configandenvshould be replaced by your configuration and environment file paths.

After creating your FastAPI app, run the following command in your terminal to get it up and running:

uvicorn main:app

---

🔄 migration from llama-index

switching from Llama-Index? We've got you covered.

👉 easy migration

>>>fromllama_indeximportStorageContext,ServiceContext,VectorStoreIndex
>>>fromllama_index.vectorstoresimportLanceDBVectorStore

>>>fromautollmimportAutoQueryEngine

>>>vector_store=LanceDBVectorStore(uri="./.lancedb")
>>>storage_context=StorageContext.from_defaults(vector_store=vector_store)
>>>service_context=ServiceContext.from_defaults()
>>>index=VectorStoreIndex.from_documents(
documents=documents,
storage_context=storage_contex,
service_context=service_context,
)

>>>query_engine=AutoQueryEngine.from_instances(index)

❓ FAQ

Q: Can I use this for commercial projects?

A: Yes, AutoLLM is licensed under GNU Affero General Public License (AGPL 3.0), which allows for commercial use under certain conditions.Contactus for more information.

---

roadmap

our roadmap outlines upcoming features and integrations to make autollm the most extensible and powerful base package for large language model applications.

• 1-lineGradioapp creation and deployment

• Budget based email notification

• Automated LLM evaluation

• Add more quickstart apps on pdf-chat, documentation-chat, academic-paper-analysis, patent-analysis and more!

---

📜 license

autollm is available under theGNU Affero General Public License (AGPL 3.0).

---

📞 contact

for more information, support, or questions, please contact:

• Email:[email protected]
• Website:SafeVideo
• LinkedIn:SafeVideo AI

---

🏆 contributing

love autollm? star the repo or contribute and help us make it even better!see ourcontributing guidelinesfor more information.

---

follow us for more!