Quick Tour: Ollama+Gemma+LangChain

1 min readFeb 23, 2024


Utilize Ollama for hosting Gemma and utilize LangChain for developing an AI application with a local model.

I think there’s no need to explain what Gemma is. Through this tutorial, you can set up a Gemma-based AI on your local machine with the assistance of Ollama and LangChain.

Ollama style

Locate the model

gemma (ollama.com)

Pull model

ollama run gemma:7b
ollama run gemma:2b

Run model

ollama run gemma:2b

Then in the terminal you can see

ollama run gemma:2b
pulling manifest
pulling c1864a5eb193... 100% ▕█████████████████████████████████████████████████████████████████████████████▏ 1.7 GB
pulling 097a36493f71... 100% ▕█████████████████████████████████████████████████████████████████████████████▏ 8.4 KB
pulling 109037bec39c... 100% ▕█████████████████████████████████████████████████████████████████████████████▏ 136 B
pulling 22a838ceb7fb... 100% ▕█████████████████████████████████████████████████████████████████████████████▏ 84 B
pulling 887433b89a90... 100% ▕█████████████████████████████████████████████████████████████████████████████▏ 483 B
verifying sha256 digest
writing manifest
removing any unused layers

After >>> you can write any question or query you want.

Check model list

ollama list

Integrate with LangChain

A runable command-line app

from langchain.callbacks.manager import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain_community.llms.ollama import Ollama

llm = Ollama(
while True:
query = input("Enter a query: ")