How to connect local small language models to store APIs

Tool calling with a local model works identically in structure to tool calling with a cloud API — the difference is only where the reasoning step happens. Your Python code defines a function (e.g., "look up order status"), describes it in a JSON schema the model can read, and the local model decides when to invoke it and with what arguments. Your code, not the model, actually executes the API call to the store platform.

Wiring a local model to a store API

1. Confirm your chosen model supports tool calling. Check Ollama's model library for the "Tools" tag — Llama 3.1, Mistral Nemo, and Qwen2.5 are commonly cited as supporting this; not every local model does.

2. Define the store lookup as a tool with a JSON schema. Describe the function's name, purpose, and parameters (e.g., order ID) in the same schema format the OpenAI API uses — Ollama's tool-calling interface follows this convention.

3. Execute the actual API call in your own code, not the model. When the model returns a tool-call request, your code validates the arguments and makes the real HTTP call to the store's API (Shopify Admin API, WooCommerce REST API) — the model never holds your store credentials.

# pip install ollama
import ollama

def get_order_status(order_id: str) -> dict:
    # Your code makes the real call to the store API here --
    # the model never sees or holds the API key
    return {"order_id": order_id, "status": "shipped"}

response = ollama.chat(
    model="llama3.1",
    messages=[{"role": "user", "content": "What's the status of order #4471?"}],
    tools=[{
        "type": "function",
        "function": {
            "name": "get_order_status",
            "description": "Look up the shipping status of a store order",
            "parameters": {
                "type": "object",
                "properties": {"order_id": {"type": "string"}},
                "required": ["order_id"]
            }
        }
    }]
)

The main practical reason to reach for a local model here is keeping the customer's query text off a third-party inference API — the store API call itself still travels over the network either way, since that's the store platform's own endpoint. Choose a local model specifically for that reasoning-privacy benefit, not under the assumption that "local" makes the entire pipeline more private end-to-end.