bestmultiagent

Published 2026-11-18

Best multi-agent frameworks for local models

SHORT ANSWER

CrewAI, Pydantic AI, Google ADK, Mastra and Microsoft Agent Framework all document a route to Ollama. CrewAI falls back to LiteLLM for providers without a native SDK. Pydantic AI has a dedicated OllamaModel class. ADK points its LiteLlm wrapper class at a local server. Mastra takes a model instance from ollama-ai-provider-v2. Microsoft's provider page lists Ollama for local open-source inference. Test tool calling on your own model first.

How does CrewAI connect to Ollama?

CrewAI reaches Ollama through LiteLLM. The docs say CrewAI uses LiteLLM as a flexible fallback for every provider outside its native integrations, and they list Ollama among LiteLLM's providers. To use a provider like that, you add LiteLLM as a dependency with uv add 'crewai[litellm]'.

For local setup, the guide has you install Ollama and then pull the desired model, for example with ollama pull llama3.2, before you configure the agent's llm. CrewAI's default model is set by the OPENAI_MODEL_NAME environment variable, and the docs say you can configure your agents to use a different model or provider. The CrewAI tool page covers the rest of the framework.

What limits does CrewAI document for non-default models?

Context window sizing is the limit most likely to affect a local model. CrewAI measures summarization and tool-result deferral against 85% of the window listed in its bundled LiteLLM model catalog. A model missing from that catalog falls back to a provider default, which is 8,192 tokens for LiteLLM-routed models, so CrewAI uses 6,963. The docs say both features start far earlier than the model's real window warrants.

To see the figure CrewAI applies, call get_context_window_size() on the LLM. The LLM is built as it would be for a run, so the CrewAI agent docs say the provider's API key must be set.

On tools, the agent docs say to ensure your chosen llm supports features such as function calling. They also suggest you set use_system_prompt: false for older models that lack system messages. The LLM docs add that structured output support varies by provider and model, and they ask you to test the model before relying on it in production.

What does Pydantic AI document for Ollama?

Pydantic AI ships OllamaModel, which needs either pydantic-ai or pydantic-ai-slim with the openai optional group. It supports self-hosted Ollama servers, running locally or remotely, and Ollama Cloud. For servers running locally, the base URL is http://localhost:11434/v1. You can also set the OLLAMA_BASE_URL environment variable. For backward compatibility, OllamaModel uses Ollama's OpenAI-compatible Chat Completions API at /v1/chat/completions. The Pydantic AI page has more on the framework.

Schema enforcement depends on where Ollama runs. Self-hosted Ollama v0.5.0 and later enforces response_format with json_schema, so NativeOutput produces schema-valid output at generation time. On Ollama Cloud, the docs say schemas are silently not enforced. Using NativeOutput with a Cloud model raises a clear UserError instead of a silent retry loop. For Cloud, the docs recommend the default ToolOutput or PromptedOutput.

How does Google ADK reach local models?

ADK reaches local models through LiteLLM. Its docs describe LiteLLM as a translation layer for models with an OpenAI-compatible interface, and they say you can run open-source models locally or self-host them and integrate them using LiteLLM. For a local host, you configure the LiteLlm wrapper class to point to your local model server. ADK requires litellm 1.84 or later.

The docs flag two caveats. In Python, when a model cannot take output_schema and tools in one request, ADK adds a set_model_response tool, which may not work reliably, and the docs suggest sub-agents that handle output formatting. The LiteLLM page also reports unauthorized code in LiteLLM versions 1.82.7 and 1.82.8 on PyPI. If you installed or upgraded LiteLLM during that period, it says to rotate all secrets and credentials.

What about the OpenAI Agents SDK with other providers?

The SDK calls itself provider-agnostic and says it supports 100+ other LLMs beyond the OpenAI APIs. Its models page lists where non-OpenAI providers need extra settings.

How do Mastra and Microsoft Agent Framework handle Ollama?

Mastra uses a separate package for Ollama. Its provider page says Ollama is available through the AI SDK, so you install the provider package (npm install ollama-ai-provider-v2). The page also says Ollama isn't a model router provider, so ollama/<model> strings don't resolve. You pass a model instance to the agent's model option instead. See the Mastra page for the wider framework.

Microsoft Agent Framework lists Ollama for local open-source model inference. Its Python provider list adds Foundry Local for supported Foundry models run locally. Beyond named providers, any service with an IChatClient implementation can back a ChatClientAgent.

Which framework should you pick for a local-model build?

On a single-framework project, the framework is the right pick. If your agents already run on one of the frameworks above, keep it and use its documented Ollama route. If you need schema-valid output from a self-hosted server, start with the Pydantic AI path described earlier.

Before you commit, run the exact model you plan to ship through a test with your real tools and output types. Each section above also names settings to change when a default does not fit a local server.

What do readers ask about best multi-agent frameworks for local models?

Do I need LiteLLM to run a local model?

That depends on the framework. Pydantic AI's OllamaModel installs with the openai optional group, and Mastra uses a model instance from ollama-ai-provider-v2. CrewAI falls back to LiteLLM for providers outside its native list.

Why does my CrewAI agent stop after repeating a tool call?

CrewAI stops the task with ToolExecutionFailedError when an agent calls the same tool with the same arguments and gets the same result five times in one run. The docs say the limit of five is fixed. A local model that loops on a tool will hit it, so check function calling support first.

Why does ADK with LiteLLM throw a UnicodeDecodeError on Windows?

The ADK docs say LiteLLM may attempt to read cached files with the default Windows encoding instead of UTF-8. Set the PYTHONUTF8 environment variable to 1 to prevent it.

Where can I compare frameworks beyond local-model support?

The rankings page lets you adjust how much each criterion counts for your own build.

Which sources does this page cite?