feat: Allow AI to deny using tools, and to use them multiple times until satisfied with its answer

[jack5github]

This has the potential to be a really intuitive way of utilising MCP servers with Ollama, but it's missing the natural flow of knowledge gathering and execution that most LLM applications strive to have. It would be great if there were settings added that allowed models to make more decisions in-between responses and thus return a more complete output.

[jonigl]

Hey @jack5github, thanks so much for reaching out 🙌 I really appreciate the kind words!

I think I understand what you're getting at: you're suggesting that when, for example, a single MCP Server tool call doesn't return enough information, the model should be able to keep iterating, kind of like an agent mode, right?

If you could share a bit more detail about what you'd like to see in practice (and maybe a few example scenarios), that would be super helpful. It’ll give us a clearer idea of how best to support this kind of autonomous behavior.

Looking forward to hearing more!

[jack5github]

I can indeed give you an example scenario. Say that we have an MCP server with tools for getting database table information, table column information, and document contents, and the database has a document table. If the user is to ask what the structure of any document is, the agent should be aware that it cannot access tool 2 without a table name, nor tool 3 without a document ID, so it should first access tool 1 to get the table names and then tool 2 to get the document structure listed in the database, both before replying. It may also express a desire to use tool 3, provided that it is given an ID of a document. Additionally, if it is asked a question unrelated to any of the tools, it should not use any of them.

When I was using this interface in a very similar scenario, I wasn't able to get the model to use tool 2. It would always try to use a tool even when it wasn't appropriate, and when I asked it what the structure of a document was, it would try to use tool 3, inserting the number 12345 in the ID field. When the tool returned an error, it wouldn't reconsider and would just create a reply based on that error.

I am also aware that there may be a lot of tools available to a model, and because of its limited context length, it may forget the system or user prompt, or lose focus on the tools. You could potentially leverage an embedding model, such as nomic-embed-text, to select the top few most similar tools, so that the model only has to consider if it wants to use anything from that selection. All of these new features would be controllable via the settings exposed by the CLI.

[jonigl]

Hey @jack5github, thanks a lot for the detailed follow-up.

That scenario makes total sense and highlights a strong use case for a potential new feature to support more flexible, agent-like tool usage. I’ve got a few ideas I’d like to explore based on what you shared.

Something that can definitely help in some cases is providing a good system prompt. Since users can customize that, it can go a long way in steering the model toward better decisions.

One thing I’ve noticed, though, is that some MCP servers return complex JSON schemas that aren’t always fully visible to the model. That makes it harder to figure out which tool to use and how. It also depends on how familiar the model is with the domain and how capable it is at using tools.

Just for context, I opened an issue and a couple of PRs in the Ollama repo to help expose full tool schemas and make things a bit less restrictive:

ollama/ollama#11444
ollama/ollama#11448
ollama/ollama-python#542

This won’t land overnight, but it’s definitely something I’m actively working toward.

Really appreciate you sharing all this.

[dstoc]

I think I understand what you're getting at: you're suggesting that when, for example, a single MCP Server tool call doesn't return enough information, the model should be able to keep iterating, kind of like an agent mode, right?

@jonigl I expected ollmcp to keep iterating until the model returned a response without any further function calls. Is that in scope for this issue or a bug?

In my use case I had list_directory and read_file functions, and multiple calls would be required to fulfill the request.

In that situation now I get a red message after the first list_directory and thinking about reading other files that says "No content response received." and no further progress.

Aside: Is there a log of inputs/outputs between ollmcp and ollama? That would help to understand what's actually happening in some situations.

[jonigl]

Hi @dstoc! Thanks for reaching out! 🙂

@jonigl I expected ollmcp to keep iterating until the model returned a response without any further function calls. Is that in scope for this issue or a bug?

This agent-mode feature is planned for future versions.

In my use case I had list_directory and read_file functions, and multiple calls would be required to fulfill the request.

You can still get multiple tool calls in a single turn from the model, and you can also drive the model through chat-style turns.

Tip

Pro tip: You can steer the model’s behavior even better by using a specific system prompt.

Here’s a demo of multiple tool calls:

In that situation now I get a red message after the first list_directory and thinking about reading other files that says "No content response received." and no further progress.
Aside: Is there a log of inputs/outputs between ollmcp and ollama? That would help to understand what's actually happening in some situations.

Yes, this is a known behavior that sometimes occurs depending on the model and the tools the model happens to use. You can check out this issue: #61, where @aitormendez has shared a detailed analysis. I’ll also be looking into this myself, but unfortunately, it’s not easy to reproduce. I’ve even debugged Ollama responses, and sometimes the content simply comes back empty when this happens.

In the meantime, could you share your ollama version, ollmcp version, the MCP servers you’re using, and the prompts you’ve tested so I can try to reproduce the error? 🙌

Greetings!