I was considering making an MCP SEP (specification enhancement proposal) — https://modelcontextprotocol.io/community/sep-guidelines, though I'm curious if other MCP tinkerers feel the issue exists, should be solved like that, etc. What do you think?
nmilo · 37m ago
I don't think this is a protocol issue, the LLMs simply weren't RLHFed to do that
vlaaad · 24m ago
Not true, structured outputs enforce output formats with 100% reliability, e.g., https://platform.openai.com/docs/guides/structured-outputs says "Structured Outputs is a feature that ensures the model will always generate responses that adhere to your supplied JSON Schema, so you don't need to worry about the model omitting a required key, or hallucinating an invalid enum value"
LudwigNagasena · 1h ago
> there is no way to tell the AI agent “for this argument, look up a JSON schema using this other tool”
There is a description field, it seems sufficient for most cases. You can also dynamically change your tools using `listChanged` capability.
vlaaad · 1h ago
Sure, but the need for accuracy will only increase; there is a difference between suggesting an LLM to put a schema in its context before calling the tool vs forcing the LLM to use a structured output returned from a tool dynamically.
We already have 100% reliable structured outputs if we are making chatbots with LLM integrations directly; I don't want to lose this.
There is a description field, it seems sufficient for most cases. You can also dynamically change your tools using `listChanged` capability.
We already have 100% reliable structured outputs if we are making chatbots with LLM integrations directly; I don't want to lose this.