Function Calling and JSON Mode

Function calling enables dynamic workflows by allowing the model to select and suggest function calls based on user input, which helps in building agentic workflows. By defining a set of functions, or tools, you provide context that lets the model recommend and fill in function arguments as needed.

How function calling works

Function calling enables adaptive workflows that leverage real-time data and structured outputs, creating more dynamic and responsive model interactions.

Submit a Query with tools: Start by submitting a user query along with available tools defined in JSON Schema. This schema specifies parameters for each function.
The model processes and suggests: The model interprets the query, assesses intent, and decides if it will respond conversationally or suggest function calls. If a function is called, it fills in the arguments based on the schema.
Receive a model response: You’ll get a response from the model, which may include a function call suggestion. Execute the function with the provided arguments and return the result to the model for further interaction.

Not all models support function calling. See Infercom Inference Service models for the list of available models and their capabilities.

When using gpt-oss-120b for function calling, set reasoning_effort to "high" for best results.

Model support

Function calling is supported on the models below. Always check Supported models for current availability.

Model ID	Region	Tools	Forced `tool_choice`	`response_format`	Tools with an image
`MiniMax-M2.7`	EU	Yes	Yes	Yes	Not applicable - text only
`gpt-oss-120b`	EU	Yes	Yes	Yes	Not applicable - text only
`gemma-4-31B-it`	EU	Yes	See the warning below	Yes	Yes
`DeepSeek-V3.1`	Non-EU	Yes	Yes	Yes	Not applicable - text only
`DeepSeek-V3.2`	Non-EU	Yes	Yes	Yes	Not applicable - text only
`Meta-Llama-3.3-70B-Instruct`	Non-EU	Yes	Yes	Yes	Not applicable - text only

Vision requests support the same tools, tool_choice and response_format options as text-only requests. The constraint is applied during generation. See Vision.

Example usage

The examples below describe each step of using function calling with an end-to-end example after the last step.

Step 1: Define the function schema

Define a JSON schema for your function. You will need to specify:

The name of the function.
A description of what it does.
The parameters, their data types, and descriptions.

Example schema for getting the weather

{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Gets the current weather information for a given city.",
    "parameters": {
      "type": "object",
      "properties": {
        "city": {
          "type": "string",
          "description": "Name of the city to get weather information for."
        }
      },
      "required": ["city"]
    }
  }
}

Step 2: Configure function calling in your request

When sending a request, include the function definition in the tools parameter and set tool_choice to the following:

auto : allows the model to choose between generating a message or calling a function. This is the default tool choice when the field is not specified.
required : This forces the model to generate a function call. The model will then always select one or more function(s) to call.
To enforce a specific function call, set tool_choice = {"type": "function", "function": {"name": "get_weather"}}. This ensures the model will only use the specified function.

A forced tool_choice is not guaranteed on every model. Two behaviours to plan for.gemma-4-31B-it does not always honour the constraint. On prompts that do not obviously call for a tool, roughly a third to a half return a plain text answer instead of the forced call. The other models in the table above are not affected. If you need a reliable forced call on Infercom’s EU infrastructure, use MiniMax-M2.7 or gpt-oss-120b.An unhonoured forced call returns HTTP 400. When the model produces no valid function call, the request fails with the body below. Note that this response does not use the standard error envelope: error is a string, not an object, and there is no request_id.

{
  "error": "Specified tool_choice with function name \"book_flight\", but the model output contains zero valid function call.",
  "error_code": null,
  "error_model_output": "I can help you book a flight from Munich to Lisbon...",
  "error_type": "Invalid function calling output."
}

The error_model_output field carries the text the model produced instead. Read it if you need to recover the answer. Handle this status in any workflow that forces a call. See API error codes.

The following code block shows a fake weather lookup that returns a random temperature between 20°C and 50°C. For accurate and real-time weather data, use a proper weather API.

import sambanova
import random
import json

# Initialize the client with the Infercom base URL and API key
client = sambanova.SambaNova(
    base_url="https://api.infercom.ai/v1",
    api_key="your-infercom-api-key"
)

MODEL = 'MiniMax-M2.7'

def get_weather(city: str) -> dict:
    """
    Fake weather lookup: returns a random temperature between 20°C and 50°C.
    """
    temp = random.randint(20, 50)
    return {
        "city": city,
        "temperature_celsius": temp
    }

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get weather of a location, the user should supply a location first",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    }
                },
                "required": ["city"]
            },
        }
    },
]

messages = [{"role": "user", "content": "What's the weather like in Paris today?"}]


completion = client.chat.completions.create(
    model=MODEL,
    messages=messages,
    tools=tools
)

print(completion)

import openai
import random
import json

# Initialize the client with the Infercom base URL and API key
client = openai.OpenAI(
    base_url="https://api.infercom.ai/v1",
    api_key="your-infercom-api-key"
)

MODEL = 'MiniMax-M2.7'

def get_weather(city: str) -> dict:
    """
    Fake weather lookup: returns a random temperature between 20°C and 50°C.
    """
    temp = random.randint(20, 50)
    return {
        "city": city,
        "temperature_celsius": temp
    }

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get weather of a location, the user should supply a location first",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    }
                },
                "required": ["city"]
            },
        }
    },
]

messages = [{"role": "user", "content": "What's the weather like in Paris today?"}]


completion = client.chat.completions.create(
    model=MODEL,
    messages=messages,
    tools=tools
)

print(completion)

Step 3: Handle tool calls

If the model chooses to call a function, you will find tool_calls in the response. Extract the function call details and execute the corresponding function with the provided parameters.

Example code

tool_call = completion.choices[0].message.tool_calls[0]
args = json.loads(tool_call.function.arguments)

result = get_weather(args["city"])

Step 4: Provide function results back to the model

Once you have computed the result, pass it back to the model to continue the conversation or confirm the output.

Example code

messages.append(completion.choices[0].message)  # append model's function call message
messages.append({                               # append result message
    "role": "tool",
    "tool_call_id": tool_call.id,
    "content": str(result)
})

completion_2 = client.chat.completions.create(
    model=MODEL,
    messages=messages,
 
)
print(completion_2.choices[0].message.content)

Step 5: Example output

An example output is shown below.

The current weather in Paris is 25 degrees Celsius.

End-to-end example

The following code block shows a fake weather lookup that returns a random temperature between 20°C and 50°C. For accurate and real-time weather data, use a proper weather API.

import sambanova
import random
import json

# Define the SambaNova client
client = sambanova.SambaNova(
    base_url="https://api.infercom.ai/v1",
    api_key="your-infercom-api-key"
)

MODEL = 'MiniMax-M2.7'

def get_weather(city: str) -> dict: 
    """
    Fake weather lookup: returns a random temperature between 20°C and 50°C.
    """
    temp = random.randint(20, 50)
    return {
        "city": city,
        "temperature_celsius": temp
    }


tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get weather of a location, the user should supply a location first",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    }
                },
                "required": ["city"]
            },
        }
    },
]

messages = [{"role": "user", "content": "What's the weather like in Paris today?"}]


completion = client.chat.completions.create(
    model=MODEL,
    messages=messages,
    tools=tools
)

print(completion)


tool_call = completion.choices[0].message.tool_calls[0]
args = json.loads(tool_call.function.arguments)

result = get_weather(args["city"])


messages.append({                               # append model's function call message
 "role": "assistant",
 "content": None,
 "tool_calls": [tool_call]
})

messages.append({                               # append result message
 "role": "tool",                           
 "tool_call_id": tool_call.id,
 "content": json.dumps(result)
})

completion_2 = client.chat.completions.create(
    model=MODEL,
    messages=messages,
 
)
print(completion_2.choices[0].message.content)

import openai
import random
import json

# Define the OpenAI client
client = openai.OpenAI(
    base_url="https://api.infercom.ai/v1",
    api_key="your-infercom-api-key"
)

MODEL = 'MiniMax-M2.7'

def get_weather(city: str) -> dict: 
    """
    Fake weather lookup: returns a random temperature between 20°C and 50°C.
    """
    temp = random.randint(20, 50)
    return {
        "city": city,
        "temperature_celsius": temp
    }


tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get weather of a location, the user should supply a location first",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    }
                },
                "required": ["city"]
            },
        }
    },
]

messages = [{"role": "user", "content": "What's the weather like in Paris today?"}]


completion = client.chat.completions.create(
    model=MODEL,
    messages=messages,
    tools=tools
)

print(completion)


tool_call = completion.choices[0].message.tool_calls[0]
args = json.loads(tool_call.function.arguments)

result = get_weather(args["city"])


messages.append({                               # append model's function call message
 "role": "assistant",
 "content": None,
 "tool_calls": [tool_call]
})

messages.append({                               # append result message
 "role": "tool",                           
 "tool_call_id": tool_call.id,
 "content": json.dumps(result)
})

completion_2 = client.chat.completions.create(
    model=MODEL,
    messages=messages,
 
)
print(completion_2.choices[0].message.content)

Streaming tool calls

Set stream: true to receive tool calls as they are generated. Arguments arrive in the tool_calls field of the delta. No reasoning scaffolding appears in delta.content.

Ignore delta fields you do not recognise. The set of keys in delta is not fixed across models.gpt-oss-120b adds a channel field to the delta on streams that carry no tools. It holds the value analysis while the model reasons. The field is absent as soon as you supply a tools array, and no other model emits it.

Request	Delta keys
`gpt-oss-120b`, no tools	`channel`, `content`, `reasoning`, `role`
`gpt-oss-120b`, tools supplied	`content`, `reasoning`, `role`, `tool_calls`
Every other model	`content`, `reasoning`, `role`, `tool_calls`

A client that validates delta keys strictly will break on the first case. Parse the fields you need and discard the rest.

JSON schema

You can set the response_format parameter to your defined schema to ensure the model produces a JSON object that matches your specified structure.

import sambanova
import json

# Define the SambaNova client
client = sambanova.SambaNova(
    base_url="https://api.infercom.ai/v1",
    api_key="your-infercom-api-key"
)

MODEL = "DeepSeek-V3.1"

# Define the schema
response_format = {
    "type": "json_schema",
    "json_schema": {
        "name": "data_extraction",
        "schema": {
            "type": "object",
            "properties": {
                "section": {
                    "type": "string"
                },
                "products": {
                    "type": "array",
                    "items": {
                        "type": "string"
                    }
                }
            },
            "required": ["section", "products"],
            "additionalProperties": False
        },
        "strict": False 
    }
}

# Call the API
completion = client.chat.completions.create(
    model=MODEL,
    messages=[
        {
            "role": "system",
            "content": "You are an expert at structured data extraction. You will be given unstructured text and should convert it into the given structure."
        },
        {
            "role": "user",
            "content": "the section 24 has appliances, and videogames"
        }
    ],
    response_format=response_format
)

# Print the parsed result
print(completion)

import openai
import json

# Define the OpenAI client
client = openai.OpenAI(
    base_url="https://api.infercom.ai/v1",
    api_key="your-infercom-api-key"
)

MODEL = "DeepSeek-V3.1"

# Define the schema
response_format = {
    "type": "json_schema",
    "json_schema": {
        "name": "data_extraction",
        "schema": {
            "type": "object",
            "properties": {
                "section": {
                    "type": "string"
                },
                "products": {
                    "type": "array",
                    "items": {
                        "type": "string"
                    }
                }
            },
            "required": ["section", "products"],
            "additionalProperties": False
        },
        "strict": False 
    }
}

# Call the API
completion = client.chat.completions.create(
    model=MODEL,
    messages=[
        {
            "role": "system",
            "content": "You are an expert at structured data extraction. You will be given unstructured text and should convert it into the given structure."
        },
        {
            "role": "user",
            "content": "the section 24 has appliances, and videogames"
        }
    ],
    response_format=response_format
)

# Print the parsed result
print(completion)

Ensure to set the "strict" parameter to false, as true isn’t supported yet. When it is available, it will ensure the model strictly follows your function schema instead of making a best-effort attempt.

JSON mode

You can set the response_format parameter to json_object in your request to ensure that the model outputs a valid JSON. In case the mode is not able to generate a valid JSON, an error will be returned.

In case the model fails to generate a valid JSON, you will get an error message Model did not output valid JSON.

import sambanova

# Define the SambaNova client
client = sambanova.SambaNova(
    base_url="https://api.infercom.ai/v1",
    api_key="your-infercom-api-key"
)

MODEL = 'MiniMax-M2.7'


def run_conversation(user_prompt):
    # Initial conversation with user input
    messages = [
        {
            "role": "system",
            "content": "Always provide the response in this JSON format: {\"country\": \"name\", \"capital\": \"xx\"}"
        },

        {
            "role": "user",
            "content": user_prompt,
        }
    ]

    # First API call to get model's response
    response = client.chat.completions.create(
        model=MODEL,
        messages=messages,
        max_tokens=500,
        response_format = { "type": "json_object"},
        # stream = True
    )
    
    response_message = response.choices[0].message
    print(response_message)


run_conversation('what is the capital of Austria')

import openai

# Define the OpenAI client
client = openai.OpenAI(
    base_url="https://api.infercom.ai/v1",
    api_key="your-infercom-api-key"
)

MODEL = 'MiniMax-M2.7'


def run_conversation(user_prompt):
    # Initial conversation with user input
    messages = [
        {
            "role": "system",
            "content": "Always provide the response in this JSON format: {\"country\": \"name\", \"capital\": \"xx\"}"
        },

        {
            "role": "user",
            "content": user_prompt,
        }
    ]

    # First API call to get model's response
    response = client.chat.completions.create(
        model=MODEL,
        messages=messages,
        max_tokens=500,
        response_format = { "type": "json_object"},
        # stream = True
    )
    
    response_message = response.choices[0].message
    print(response_message)


run_conversation('what is the capital of Austria')

Example response

ChatCompletionMessage(content='{"country": "Austria", "capital": "Vienna"}', role='assistant', function_call=None, tool_calls=None)

Other methods of structured outputs

Beyond JSON mode, structured outputs can also be generated using the Instructor library, which provides a convenient wrapper for extracting structured data from LLM responses using Pydantic models.

Get started

Models

Features

Agentic Coding

Build

Resources

Implement Function Calling and JSON Mode Features - Developer Guide

How function calling works

Model support

Example usage

Step 1: Define the function schema

Step 2: Configure function calling in your request

Step 3: Handle tool calls

Step 4: Provide function results back to the model

Step 5: Example output

End-to-end example

Streaming tool calls

JSON schema

JSON mode

Other methods of structured outputs

​How function calling works

​Model support

​Example usage

​Step 1: Define the function schema

​Step 2: Configure function calling in your request

​Step 3: Handle tool calls

​Step 4: Provide function results back to the model

​Step 5: Example output

​End-to-end example

​Streaming tool calls

​JSON schema

​JSON mode

​Other methods of structured outputs

How function calling works

Model support

Example usage

Step 1: Define the function schema

Step 2: Configure function calling in your request

Step 3: Handle tool calls

Step 4: Provide function results back to the model

Step 5: Example output

End-to-end example

Streaming tool calls

JSON schema

JSON mode

Other methods of structured outputs