support multiple tool calls in output #22
Conversation
|
Many thanks for looking into this one @erikgaas ! :) Can you add a couple of tests, including >=1 with multiple tool calls -- something that would have failed before. Once we have that, maybe we can investigate alternate ways of implementing this. The current one is too complex for my little brain to comprehend! (Possibly that's my brain's fault; if we can't find something simpler but still correct, then so be it) |
|
@RensDimmendaal fyi I think your last PR to make aformat_stream prettier didn't sync right between nbs and code. I'll be careful about this to make sure nothing crazy happens. |
| yield f"\n<details class='tool-usage-details'>\n\n `{fn.name}({_trunc_str(fn.arguments)})`\n" | ||
| elif isinstance(o, dict) and 'tool_call_id' in o: | ||
| yield f" - `{_trunc_str(_clean_str(o.get('content')))}`\n\n</details>\n\n" | ||
| fmt = mk_stream_formatter() |
There was a problem hiding this comment.
@jph00 @RensDimmendaal What do you think of this factored out aformat_stream?
There was a problem hiding this comment.
Next step is to step through all of this with synthetic data. I can do that here, but will more likely continue that in the other PR.
There was a problem hiding this comment.
@erikgaas wdyt of this was an alternative? its 12 lines more concise. And a bit more understandable, for me at least. I got a bit confused w all the nested functions and the hidden state.
#| export
class StreamFormatter:
def __init__(self): self.n=0
def __call__(self, o):
if isinstance(o, ModelResponseStream): return delta_text(o, show_tools=False)
if isinstance(o, ModelResponse):
if not (tcs := getattr(o.choices[0].message, 'tool_calls', None)): return ''
self.n = len(tcs)
calls = '\n'.join([f" `{tc.function.name}({_trunc_str(tc.function.arguments)})`" for tc in tcs])
return f"\n<details class='tool-usage-details'>\n\n{calls}\n"
if isinstance(o, dict) and 'tool_call_id' in o:
self.n -= 1
return f" - `{_trunc_str(_clean_str(o.get('content')))}`\n" + ('' if self.n else '\n</details>\n\n')
return ''
async def aformat_stream(rs):
fmt = StreamFormatter()
async for o in rs: yield fmt(o)
| res = ''.join(f"🔧 {tc.function.name}" for tc in delta.tool_calls if tc.id and tc.function.name) | ||
| if res: return f'\n{res}\n' | ||
| if hasattr(delta,'reasoning_content'): return '🧠' if delta.reasoning_content else '\n\n' | ||
| return None |
There was a problem hiding this comment.
return None is not needed
| def delta_text(msg, show_tools=True): | ||
| "Extract printable content from streaming delta, return None if nothing to print" | ||
| c = msg.choices[0] | ||
| if not c: return c |
There was a problem hiding this comment.
could save a line w if not (c:=msg.choices[0]): return c
| # %% ../nbs/00_core.ipynb | ||
| def _fmt_tool_call(o): | ||
| if not (c := getattr(o.choices[0].message, 'tool_calls', None)): return '' | ||
| calls = '\n'.join([f" `{tc.function.name}({_trunc_str(tc.function.arguments)})`" for tc in c]) |
There was a problem hiding this comment.
Ive have some issues in the past relying on tc.function.name being accessible as attributes...and instead having to access them via the [...] index. Sorry that I cant be more specific. But I suggest testing this one well as I've been burnt by that one in the past.
|
The markdown formatting looks a bit off, w first the multiple toolcalls on one line and then each result in a list. 
|
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "assert md.count('<detail') == md.count('</detail')" |
| "{'tool_call_id': 'toolu_017xC4A71CAwU2e27gjU9C7X',\n", | ||
| " 'role': 'tool',\n", | ||
| " 'name': 'simple_add',\n", | ||
| " 'content': '3'}" |
There was a problem hiding this comment.
maybe you can turn these last three cells w print statements into a test, or add some note cells explaining what should we expect to see here?
|
Thanks a lot @erikgaas ! |
|
@erikgaas also this one! :) |
|
@RensDimmendaal is working on this in a different direction now so closing this |
3 participants
A user reported that multiple tool calls was failing to display correctly in the output cell: https://discord.com/channels/1303610061167263814/1310989166875508849/1423065780433719449
This was because lisette assumed a model response could only contain 1 tool call and therefore only would require one tool result.
This PR counts the number of tools made and waits for all of the results to come in before providing the closing tag.