Context overload. #coffeesoftware #RonaldDehuy watch more at www.youtube.com/live/RbkpxQW...
"So, the way tools work in AI models is you send, in effect, extra metadata about which tools are available to the model, and normally you send every single tool that you've got. If you've got 20 different tools, that's 20 different paragraphs explaining each time what tools are available, and the user may not even ask for anything related to those tools. So, the question is, can we give the model access to these tools without having to send the redundant definition of those tools each time, because that consumes tokens and that costs money, or the very worst CPU in time, right? And it can also muddy the results. If you give a model too much data, it won't be focused on one particular thing. It's just like you. If, like, you say, hey, I need you to answer the five following questions. Yeah, yeah, yeah. Too much. It's too much. Context overload. Literally, context overload, yeah."
💬 Discussion
Context overload. #coffeesoftware #RonaldDehuy watch more at www.youtube.com/live/RbkpxQW...