Enlarge / An artist’s impression of a robot hand using a desktop calculator. (credit: Aurich Lawson | Getty Images)
Language models like ChatGPT have revolutionized the field of natural language processing, but they still struggle with some basic tasks such as arithmetic and fact-checking. Last Thursday, researchers from Meta revealed Toolformer, an AI language model that can teach itself to use external tools such as search engines, calculators, and calendars without sacrificing its core language modeling abilities.
The key to Toolformer is that it can use APIs (application programming interfaces), which are a set of protocols that allow different applications to communicate with one another, often in a seamless and automated manner. During training, researchers gave Toolformer a small set of human-written examples demonstrating how each API is used and then allowed it to annotate a large language modeling dataset with potential API calls. It did this in a “self-supervised” way, meaning that it could learn without needing explicit human guidance.
The model learned to predict each text-based API call as if they were any other form of text. When in operation—generating text as the result of a human input—it can insert the calls when needed. Moreover, Toolformer can “decide” for itself which tool to use for the proper context and how to use it.