Skip to main content
Uploading static sources as training data for your AI Agents is well and good, but what if your database is huge, structured, externally-hosted, or updated in real-time? There is no way to dump the entire database into GPT Chatbot’s static knowledge library and still maintain a live link… This is where function-calling comes in. Through function-calling, you can serve data on-demand to your AI Agent in GPT Chatbot during a live conversation session. To show you how, we walk you through an example of RAG enrichment using (abstracts of) academic papers sourced from a trusted academic papers aggregator: Semantic Scholar API.

​
Setting up and testing the function

First, you’ll need an API key from your external data provider, whoever that may be, assuming they have a secure API. In our case, we requested one from Semantic Scholar directly. Since we want to enrich our LLM query response with relevant information from academic research, we need to find the appropriate API endpoint to conduct the search. Semantic Scholar offers nice documentation in that regard.
So what does this API endpoint respond with? To inspect the output, we ran a script that makes a request to the API based on hardcoded input parameters. The source code of our script is included below. You will need your own Semantic Scholar API key if you wish to run it yourself. For our example, we ran a search for relevant papers on the topic of “Multifidelity Optimization” and “Gaussian Processes”.
The output we received looks like follows:
The Semantic Scholar function returned the top 20 matching results ranked by relevance based on our input parameter specification. As you see, the response can get quite long, so we truncated the output for conciseness. Ideally, you want a structured JSON response from the function’s output like what we showed above. We ran our own analysis for this particular API endpoint (with the specific set of output fields we requested) and found that on average, each returned result is between 400-500 tokens long. This means that depending on the token limit we reserved for the function output, we can only fit so many search results in before running out of space. Keep this in mind as you prioritize the information you wish to supply to the LLM during RAG. Now we know what the LLM would see as additional context provided by the function call, we can start linking it to our GPT Chatbot Agent.
It is ALWAYS advisable to write your own script and test for the output of the API endpoint first. You want to know exactly what kind information is being fed into your AI Agent.

​
Create and prepare the AI Agent

In your chatbot inside GPT Chatbot, you must first create an appropriate AI Agent that you plan to give this function-calling capability to. In our example, we created “The Professor”. We then defined an associated Agent description and base prompt.
We picked the GPT-4-0125-8k Model for this example. You can see our simple prompt in the screenshot below.
Next, we save the Agent and go to the “Knowledge” tab to disable static RAG from the chatbot’s own knowledge library. This is only necessary if you don’t want to use any training data from the static sources list. In our example, we didn’t upload any training data anyways, but we do this as standard practice to keep things clean.
Save the Agent.

​
Function setup

Within your Agent, head over to the “Functions” tab. Change the “Response Context Limit” to the maximum allowed. Then click “Add function”.
This is where you tell the LLM exactly what the function does and how it works. The LLM will then decide on its own whether it needs the function when responding to the user during a conversation, then call it with the appropriate parameters as needed.
The function name and description are particularly useful in helping the AI understand what the function does. Please make sure to be as explicit as possible in your own definition. The function description must be less than 1024 characters, including spaces. In our case, we wrote the following:
Next, define the API endpoint. The Semantic Scholar endpoint we are using is simple and straighforward:
After that, choose the HTTP method. We use “GET” for our example. The specific API endpoint you are using should have documentation about this distinction.

​
Fixed parameters

Static (Fixed) Parameters remain constant across all API requests. They are pre-defined and reflect settings or configurations that do not change with each call. For example, parameters that specify the format of the response or that enable/disable certain features globally across all API interactions fall into this category. The term “static” emphasizes their unchanging nature. Depending on your particular API endpoint, you may need to append some fixed parameters in the URL. The API we use above from Semantic Scholar does not require fixed parameters. But if you are using a more complex endpoint like this: https://app.outscraper.com/api-docs#tag/Businesses-and-POI/paths/~1maps~1search-v3/get, then your API endpoint may include fixed parameters and look something like:
Notice that at the end of “https://api.app.outscraper.com/maps/search-v3”, we get into fixed parameters. This is where you define parameters that will not change when the function is called. Always define fixed parameters this way!
If your API is capable of returning a more concise response by configuring certain parameters (i.e., tell the API to leave out unnecessary metadata in the response), then we highly suggest that you do so. It will reduce token waste and help AI understand the context better. In the above example, we set “fields=name,full_address,phone,site” specifically to limit the response so that it contains only name, full_address, phone, and site information - and nothing else.

​
Headers and Authentication

In general, secure public-facing APIs require authentication. Semantic Scholar is the same. In the Header, we provide our API key. The specific key field used to supply your authentication token will differ based on the particular API you use, so please make sure to be informed on your API provider’s documentation. For our example, the key field is named “x-api-key”:

​
Variable parameters

Finally, we get to variable parameters. Variable (Dynamic) Parameters dynamically change based on user input or conversation context, unlike their static counterparts. In function calling, these parameters adapt to the specifics of each request, being determined at runtime. This ensures that API calls made by the AI are tailored to the immediate needs of the conversation, enabling a more personalized and relevant interaction. The AI extracts these parameters directly from the dialogue, determining when and how to call the function based on the ongoing conversation. For our function, we define the following variable parameters:
Notice that this is a structured JSON format. Under the key field “type” of the entire parameter-set, we put “object”. You should do the same for yours. Under properties, you define each parameter along with their type and description. Take “query” for example. It has type String, meaning its data type is free text. Types can be “String”, “integer”, “boolean”, “array”, etc. The description is more open-ended, but also extremely important. This is where you must tell the LLM what this parameter represents. To the best of your ability, provide default values and examples of actual parameter values when the function is called. Try to be as explicit as possible. Remember that if the AI has to guess whether a parameter named ‘language’ should take ‘en’ or ‘english’ as input, chances are it will guess wrong, and you will see errors in your chatbot output. Depending on the function you are working with, input parameters can get pretty complex. Below is another example for supplying parameters to a function (NOT related to our Semantic Scholar example) that uses an array:
The function can only be called once the “required” parameters have been collected. If necessary information is missing, the function cannot be invoked. You must define which parameters are mandatory in order for the function to be called. For additional information on the supported JSON schema, please reference this guide: https://json-schema.org/understanding-json-schema/reference/type

​
Testing the chatbot

Once everything is set up, save your function, then save the Agent. You can then test it out under “Preview” tab:
You can turn on the debug mode with the toggle on the top right corner of the screen. This will allow you to inspect the output of the function if it had been called during the LLM response:
Just like that, you equipped your AI Agent with on-demand information from an external data provider. In the future, we will also write a guide that helps you implement and host your own database search function that then gets integrated with GPT Chatbot.