Get up and running with large language models.
curl -fsSL https://ollama /install.sh | sh
The officialOllama Docker imageollama/ollama
is available on Docker Hub.
To run and chat withLlama 3.1:
ollama run llama3.1
Ollama supports a list of models available onollama /library
Here are some example models that can be downloaded:
Model | Parameters | Size | Download |
---|---|---|---|
Llama 3.1 | 8B | 4.7GB | ollama run llama3.1 |
Llama 3.1 | 70B | 40GB | ollama run llama3.1:70b |
Llama 3.1 | 405B | 231GB | ollama run llama3.1:405b |
Phi 3 Mini | 3.8B | 2.3GB | ollama run phi3 |
Phi 3 Medium | 14B | 7.9GB | ollama run phi3:medium |
Gemma 2 | 2B | 1.6GB | ollama run gemma2:2b |
Gemma 2 | 9B | 5.5GB | ollama run gemma2 |
Gemma 2 | 27B | 16GB | ollama run gemma2:27b |
Mistral | 7B | 4.1GB | ollama run mistral |
Moondream 2 | 1.4B | 829MB | ollama run moondream |
Neural Chat | 7B | 4.1GB | ollama run neural-chat |
Starling | 7B | 4.1GB | ollama run starling-lm |
Code Llama | 7B | 3.8GB | ollama run codellama |
Llama 2 Uncensored | 7B | 3.8GB | ollama run llama2-uncensored |
LLaVA | 7B | 4.5GB | ollama run llava |
Solar | 10.7B | 6.1GB | ollama run solar |
Note
You should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.
Ollama supports importing GGUF models in the Modelfile:
-
Create a file named
Modelfile
,with aFROM
instruction with the local filepath to the model you want to import.FROM./vicuna-33b.Q4_0.gguf
-
Create the model in Ollama
ollama create example -f Modelfile
-
Run the model
ollama run example
See theguideon importing models for more information.
Models from the Ollama library can be customized with a prompt. For example, to customize thellama3.1
model:
ollama pull llama3.1
Create aModelfile
:
FROM llama3.1
# set the temperature to 1 [higher is more creative, lower is more coherent]
PARAMETER temperature 1
# set the system message
SYSTEM "" "
You are Mario from Super Mario Bros. Answer as Mario, the assistant, only.
"""
Next, create and run the model:
ollama create mario -f./Modelfile
ollama run mario
>>> hi
Hello! It's your friend Mario.
For more examples, see theexamplesdirectory. For more information on working with a Modelfile, see theModelfiledocumentation.
ollama create
is used to create a model from a Modelfile.
ollama create mymodel -f./Modelfile
ollama pull llama3.1
This command can also be used to update a local model. Only the diff will be pulled.
ollama rm llama3.1
ollama cp llama3.1 my-model
For multiline input, you can wrap text with"""
:
>>> "" "Hello,
... world!
..." ""
I'm a basic program that prints the famous "Hello, world!" message to the console.
ollama run llava "What's in this image? /Users/jmorgan/Desktop/smile.png"
The image features a yellow smiley face, which is likely the central focus of the picture.
$ ollama run llama3.1 "Summarize this file: $(cat README.md)"
Ollama is a lightweight, extensible framework for building and running language models on the local machine. It provides a simple API for creating, running, and managing models, as well as a library of pre-built models that can be easily used in a variety of applications.
ollama show llama3.1
ollama list
ollama serve
is used when you want to start ollama without running the desktop application.
See thedeveloper guide
Next, start the server:
./ollama serve
Finally, in a separate shell, run a model:
./ollama run llama3.1
Ollama has a REST API for running and managing models.
curl http://localhost:11434/api/generate -d '{
"model": "llama3.1",
"prompt": "Why is the sky blue?"
}'
curl http://localhost:11434/api/chat -d '{
"model": "llama3.1",
"messages": [
{ "role": "user", "content": "why is the sky blue?" }
]
}'
See theAPI documentationfor all endpoints.
- Open WebUI
- Enchanted (macOS native)
- Hollama
- Lollms-Webui
- LibreChat
- Bionic GPT
- HTML UI
- Saddle
- Chatbot UI
- Chatbot UI v2
- Typescript UI
- Minimalistic React UI for Ollama Models
- Ollamac
- big-AGI
- Cheshire Cat assistant framework
- Amica
- chatd
- Ollama-SwiftUI
- Dify.AI
- MindMac
- NextJS Web Interface for Ollama
- Msty
- Chatbox
- WinForm Ollama Copilot
- NextChatwithGet Started Doc
- Alpaca WebUI
- OllamaGUI
- OpenAOE
- Odin Runes
- LLM-X(Progressive Web App)
- AnythingLLM (Docker + MacOs/Windows/Linux native app)
- Ollama Basic Chat: Uses HyperDiv Reactive UI
- Ollama-chats RPG
- QA-Pilot(Chat with Code Repository)
- ChatOllama(Open Source Chatbot based on Ollama with Knowledge Bases)
- CRAG Ollama Chat(Simple Web Search with Corrective RAG)
- RAGFlow(Open-source Retrieval-Augmented Generation engine based on deep document understanding)
- StreamDeploy(LLM Application Scaffold)
- chat(chat web app for teams)
- Lobe ChatwithIntegrating Doc
- Ollama RAG Chatbot(Local Chat with multiple PDFs using Ollama and RAG)
- BrainSoup(Flexible native client with RAG & multi-agent automation)
- macai(macOS client for Ollama, ChatGPT, and other compatible API back-ends)
- Olpaka(User-friendly Flutter Web App for Ollama)
- OllamaSpring(Ollama Client for macOS)
- LLocal.in(Easy to use Electron Desktop Client for Ollama)
- AiLama(A Discord User App that allows you to interact with Ollama anywhere in discord )
- Ollama with Google Mesop(Mesop Chat Client implementation with Ollama)
- Painting Droid(Painting app with AI integrations)
- Kerlig AI(AI writing assistant for macOS)
- AI Studio
- Sidellama(browser-based LLM client)
- LLMStack(No-code multi-agent framework to build LLM agents and workflows)
- BoltAI for Mac(AI Chat Client for Mac)
- Harbor(Containerized LLM Toolkit with Ollama as default backend)
- Go-CREW(Powerful Offline RAG in Golang)
- PartCAD(CAD model generation with OpenSCAD and CadQuery)
- Ollama4j Web UI- Java-based Web UI for Ollama built with Vaadin, Spring Boot and Ollama4j
- PyOllaMx- macOS application capable of chatting with both Ollama and Apple MLX models.
- Claude Dev- VSCode extension for multi-file/whole-repo coding
- Cherry Studio(Desktop client with Ollama support)
- ConfiChat(Lightweight, standalone, multi-platform, and privacy focused LLM chat interface with optional encryption)
- Archyve(RAG-enabling document library)
- crewAI with Mesop(Mesop Web Interface to run crewAI with Ollama)
- oterm
- Ellama Emacs client
- Emacs client
- gen.nvim
- ollama.nvim
- ollero.nvim
- ollama-chat.nvim
- ogpt.nvim
- gptel Emacs client
- Oatmeal
- cmdh
- ooo
- shell-pilot
- tenere
- llm-ollamaforDatasette's LLM CLI.
- typechat-cli
- ShellOracle
- tlm
- podman-ollama
- gollama
- Ollama eBook Summary
- MindsDB(Connects Ollama models with nearly 200 data platforms and apps)
- chromem-gowithexample
- LangChainandLangChain.jswithexample
- Firebase Genkit
- crewAI
- LangChainGowithexample
- LangChain4jwithexample
- LangChainRustwithexample
- LlamaIndex
- LiteLLM
- OllamaFarm for Go
- OllamaSharp for.NET
- Ollama for Ruby
- Ollama-rs for Rust
- Ollama-hpp for C++
- Ollama4j for Java
- ModelFusion Typescript Library
- OllamaKit for Swift
- Ollama for Dart
- Ollama for Laravel
- LangChainDart
- Semantic Kernel - Python
- Haystack
- Elixir LangChain
- Ollama for R - rollama
- Ollama for R - ollama-r
- Ollama-ex for Elixir
- Ollama Connector for SAP ABAP
- Testcontainers
- Portkey
- PromptingTools.jlwith anexample
- LlamaScript
- Gollm
- Ollamaclient for Golang
- High-level function abstraction in Go
- Ollama PHP
- Enchanted
- Maid
- ConfiChat(Lightweight, standalone, multi-platform, and privacy focused LLM chat interface with optional encryption)
- Raycast extension
- Discollama(Discord bot inside the Ollama discord channel)
- Continue
- Obsidian Ollama plugin
- Logseq Ollama plugin
- NotesOllama(Apple Notes Ollama plugin)
- Dagger Chatbot
- Discord AI Bot
- Ollama Telegram Bot
- Hass Ollama Conversation
- Rivet plugin
- Obsidian BMO Chatbot plugin
- Cliobot(Telegram bot with Ollama support)
- Copilot for Obsidian plugin
- Obsidian Local GPT plugin
- Open Interpreter
- Llama Coder(Copilot alternative using Ollama)
- Ollama Copilot(Proxy that allows you to use ollama as a copilot like Github copilot)
- twinny(Copilot and Copilot chat alternative using Ollama)
- Wingman-AI(Copilot code and chat alternative using Ollama and Hugging Face)
- Page Assist(Chrome Extension)
- Plasmoid Ollama Control(KDE Plasma extension that allows you to quickly manage/control Ollama model)
- AI Telegram Bot(Telegram bot using Ollama in backend)
- AI ST Completion(Sublime Text 4 AI assistant plugin with Ollama support)
- Discord-Ollama Chat Bot(Generalized TypeScript Discord Bot w/ Tuning Documentation)
- Discord AI chat/moderation botChat/moderation bot written in Python. Uses Ollama to create personalities.
- Headless Ollama(Scripts to automatically install ollama client & models on any OS for apps that depends on ollama server)
- vnc-lm(A containerized Discord bot with support for attachments and web links)
- LSP-AI(Open-source language server for AI-powered functionality)
- llama.cppproject founded by Georgi Gerganov.