Ollama

Ollama makes running open-weight models locally straightforward. Pull a model, run inference — no Python environments, no GPU configuration headaches, no cloud API costs. The model library covers everything from small embedding models to capable reasoning models.

For RAG pipelines, it's indispensable. Local embeddings via nomic-embed-text process documents without per-token costs or rate limits. Local inference with models like Gemma keeps sensitive data on-premises. We use Ollama for document processing where the volume makes API pricing impractical and the data cannot leave the network.