Function calling

In addition to providing answers to customers' questions, Markprompt is able to detect underlying intents and trigger code accordingly, such as to create a case. This is a concept know as function calling. The AI can detect a user's intent (such as "I want to speak with an agent"), make sure the user provides all the required details for the function to run, and display an adequate UI, e.g. to confirm an action.

Anatomy of a function

Here is a basic example of a function definition:

1{
2  "name": "createCase",
3  "description": "Creates a case automatically when the user asks to create a ticket/case or when they ask to speak to someone."
4}

If consists of two mandatory fields:

A function name
A function description

The description is important. This is how the LLM is able to match a user's intent with a function call. Make it as clear and explicit as possible, and keep it concise.

Functions with parameters

Some functions require additional parameters. For instance, if the user is asking to cancel their subscription, the function may need to know about the end date:

1{
2  "name": "processRefund",
3  "description": "Cancel auto-renewal and process a refund",
4  "parameters": {
5    "type": "object",
6    "properties": {
7      "cancellationDate": {
8        "type": "string",
9        "description": "The date at which the auto-renewal should end"
10      }
11    }
12    "required": ["cancellationDate"]
13  }
14}

In this example, you will notice that the cancellationDate field is marked as required. Thus, if a user asks the question:

I want to cancel my renewal and get a refund

the AI will ask a follow-up question, inquiring about the desired cancellation date:

Sorry to see you go! When would you like your subscription to end?

Triggering a function call

Now that the function's intent has been specified, it is time to bind it to a function, which will be invoked when the intent is detected.

Using the SDK

The Markprompt React SDK provides helpers to easily setup callback functions that get triggered when a function call intent is detected. Here is an example using the ChatProvider component:

1import { useCallback } from 'react';
2import { Markprompt, ChatProvider, ChatViewMessage } from '@markprompt/react';
3
4const Home = ({ children }) => {
5  const submitCase = useCallback(() => {
6    // Create a case
7    // ...
8  }, []);
9
10  return (
11    <ChatProvider
12      projectKey="<YOUR-PROJECT-KEY>"
13      chatOptions={{
14        toolChoice: 'auto',
15        tools: [
16          {
17            tool: {
18              type: 'function',
19              function: {
20                name: 'createCase',
21                description:
22                  'Creates a case automatically when the user asks to create a ticket/case or when they ask to speak to someone.',
23              },
24            },
25            call: async () => {
26              // This code gets called when the user confirms the call
27              submitCase();
28              return 'Generating case details for you.';
29            },
30          },
31        ],
32      }}
33    >
34      {children}
35    </ChatProvider>
36  );
37};

Alternatively, function calls can be set up in the Markprompt dashboard via assistants.

Showing a confirmation UI

In some situations, it may not be desirable to automatically call the function without an explicit user confirmation. The SDK allows you to define a confirmation UI to show before the function is actually called. This is done via the ToolCallsConfirmation prop. Here is an example:

1import { useCallback } from 'react';
2import { Markprompt, ChatProvider, ChatViewMessage } from '@markprompt/react';
3
4const Home = ({ children }) => {
5  // ...
6
7  return (
8    <ChatProvider
9      chatOptions={{
10        // ...
11        tools: [ ... ],
12        ToolCallsConfirmation: ({ toolCalls, toolCallsStatus, confirmToolCalls }) => {
13          // Define a custom UI allowing the user to confirm their intent
14          const toolCall = toolCalls[0];
15          if (!toolCall) {
16            return <></>;
17          }
18
19          const status = toolCallsStatus[toolCall.id]?.status;
20          return (
21            <>
22              <p>
23                Please confirm that you want to create a case:
24              </p>
25              <button
26                onClick={confirmToolCalls}
27                disabled={status === 'done'}
28              >
29                Confirm
30              </button>
31            </>
32          );
33        },
34      }}
35    >
36      {children}
37    </ChatProvider>
38  );
39};

Using the API

The Markprompt API also supports function calling. Instead of calling code, it will return a fully-specified JSON object with the required information to take adequate action:

1curl https://api.markprompt.com/chat \
2  -X POST \
3  -H "Authorization: Bearer <TOKEN>" \
4  -H "Content-Type: application/json" \
5  -H "X-Markprompt-API-Version: 2024-05-21" \
6  -d '{
7    "messages": [
8      { "content": "I need to speak with an agent", "role": "user" }
9    ],
10    "toolChoice": "auto",
11    "tools": [
12      {
13        "tool": {
14          "type": "function",
15          "function": {
16            "name": "createCase",
17            "description": "Creates a case automatically when the user asks to create a ticket/case or when they ask to speak to someone.",
18          }
19        }
20      }
21    ]
22  }'

If a function call intent is detected, the response will include a toolCalls entry:

1{
2  "toolCalls": [
3    {
4      "id": "call_id",
5      "type": "function",
6      "function": {
7        "name": "createCase"
8      }
9    }
10  ]
11}

Biasing behavior towards function calling

Markprompt can be instructed to favor function calling over responding in text. For instance, if the user has an urgent billing issue and you would like this to trigger a case creation without the user explicitly asking for it, you can provide additional system prompt instructions stating this behavior. For instance:

You must always prioritize creating a function call over returning a text response if the user enters a request that a function can handle.

If the user has an urgent billing issue, immediately trigger a call to the createCase function.

Examples

Below are some examples making use of function calling:

New ticket

Describe your issue

I am not able to find the cancellation policy if I can't attend a class I've booked. Are there any penalties or fees if I can't attend a booked class?

Ticket deflector

A form that answers questions and fills itself.

View demo View source

Is there any way I can get a refund for the unused month of October?

Yes, you are eligible for a full refund for the month of October. Do you want to proceed with the refund?

A chatbot that triggers function calls, such as reimbursements, cancelations, or customer information lookup.

View source