Understanding LLMs, Ollama, and Inference

Before building AI applications, we need to understand three fundamental concepts:

LLM
↓
Ollama
↓
Inference

What is an LLM?

LLM stands for Large Language Model.

Examples:

A language model predicts the next piece of text.

Example:

Input:

The capital of France is

Prediction:

Paris

Every response from an LLM is generated one token at a time.

Training vs Inference

Two terms you’ll hear frequently:

Training

The model learns patterns.

Books
Code
Articles
↓
Training
↓
Model

Inference

The model answers questions.

Question
↓
Model
↓
Answer

As AI application engineers, we mostly perform inference.

What is Ollama?

Think of Ollama as a runtime.

Java
↓
JVM

Python
↓
Interpreter

LLM
↓
Ollama

Java
↓
JVM

Python
↓
Interpreter

LLM
↓
Ollama

Ollama loads and runs models on your machine.

Example:

ollama run phi3:mini

Calling a Model

Once Ollama is running:

import ollama

response = ollama.chat(
    model="phi3:mini",
    messages=[
        {
            "role": "user",
            "content": "What is Kotlin?"
        }
    ]
)

print(response["message"]["content"])

Flow:

Python
↓
Ollama
↓
Model
↓
Response

Experiment

Try:

What is Android?

Then:

Explain Android to a beginner.

Notice how the model changes its answer based on the input.