local tools

tool_calls.md

LiteLLM - Best as a proxy/router

Unified interface across providers Supports tool_choice parameter Can route to local models (vLLM, Ollama) or cloud APIs Automatic format translation

Run Ollama as the backend, then put LiteLLM in front of it to add proper tool_choice support:

# Terminal 1: Start Ollama
ollama serve

# Terminal 2: LiteLLM proxy
litellm --model ollama/qwen2.5-coder:7b --api_base http://localhost:11434

use a virtual environment:

python3 -m venv path/to/venv
source path/to/venv/bin/activate
python3 -m pip install xyz

# or use pipx:

pipx install "litellm[proxy]" --python python3.12

ensure the model is available in ollama

ollama pull qwen3:14b-q4_K_M

ollama ls

litellm --model ollama/qwen3:14b-q4_K_M --api_base http://localhost:11434