Implement Vision and Multimodal Features - Developer Guide

Infercom provides access to vision-capable models on our EU sovereign infrastructure, allowing you to process both text and images with full data sovereignty. These models analyze images and generate context-aware text responses.

Supported models

Model	Context	Region	Notes
gemma-4-31B-it	128K	EU	Google’s Gemma 4 31B with vision capabilities

Note

gemma-4-31B-it runs on Infercom’s sovereign infrastructure in Germany. Your image data never leaves the EU.

Make a query with an image

On Infercom, the vision model request follows OpenAI’s multimodal input format which accepts both text and image inputs in a structured payload. While the call is similar to Text Generation, it differs by including an encoded image file, referenced via the image_path variable. A helper function is used to convert this image into a base64 string, allowing it to be passed alongside the text in the request.

1. Step 1

   Make a new Python file and copy the code below.

   Note

   This example uses gemma-4-31B-it, Google’s vision-capable Gemma 4 model hosted on Infercom’s EU sovereign infrastructure.

   Python (SambaNova SDK)

   from sambanova import SambaNova
   import base64
   
   client = SambaNova(
       base_url="https://api.infercom.ai/v1",
       api_key="your-infercom-api-key",
   )
   
   # Helper function to encode the image
   def encode_image(image_path):
     with open(image_path, "rb") as image_file:
       return base64.b64encode(image_file.read()).decode('utf-8')
   
   # The path to your image
   image_path = "sample.JPEG"
   
   # The base64 string of the image
   image_base64 = encode_image(image_path)
   
   print(image_base64)
   
   response = client.chat.completions.create(
       model="gemma-4-31B-it",
       messages=[
           {
               "role": "user",
               "content": [
                   {"type": "text", "text": "What is happening in this image?"},
                   {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_base64}"}}
               ]
           }
       ]
   )
   
   print(response.choices[0].message.content)

   Python (OpenAI)

   from openai import OpenAI
   import base64
   
   client = OpenAI(
       base_url="https://api.infercom.ai/v1",
       api_key="your-infercom-api-key",
   )
   
   # Helper function to encode the image
   def encode_image(image_path):
     with open(image_path, "rb") as image_file:
       return base64.b64encode(image_file.read()).decode('utf-8')
   
   # The path to your image
   image_path = "sample.JPEG"
   
   # The base64 string of the image
   image_base64 = encode_image(image_path)
   
   print(image_base64)
   
   response = client.chat.completions.create(
       model="gemma-4-31B-it",
       messages=[
           {
               "role": "user",
               "content": [
                   {"type": "text", "text": "What is happening in this image?"},
                   {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_base64}"}}
               ]
           }
       ]
   )
   
   print(response.choices[0].message.content)

2. Step 2

   Use your Infercom API key from the API keys and URLs page to replace the placeholder "your-infercom-api-key" in the construction of the client.

3. Step 3

   Select an image and move it to a suitable path that you can specify in the lines.

   # The path to your image
   image_path = "sample.JPEG"

4. Step 4

   Verify the prompt to pair with the image in the content portion of the user prompt.

5. Step 5

   Run the Python file to receive the text output.

Tools and structured output with images

Vision requests support the same tools, tool_choice and response_format options as text-only requests. The constraint is applied during generation, so a forced tool call or a strict JSON schema is honoured on a request that carries an image.

Option	Behaviour on a request with an image
tool_choice with a function name	The model returns that tool call. finish_reason is tool_calls
tool_choice: "none"	The model returns plain text. finish_reason is stop
response_format: {"type": "json_schema", ...}	The model returns JSON that matches the schema

Python (OpenAI)

from openai import OpenAI
import base64

client = OpenAI(
    base_url="https://api.infercom.ai/v1",
    api_key="your-infercom-api-key",
)

with open("sample.JPEG", "rb") as image_file:
    image_base64 = base64.b64encode(image_file.read()).decode("utf-8")

tools = [{
    "type": "function",
    "function": {
        "name": "report_colors",
        "parameters": {
            "type": "object",
            "properties": {"colors": {"type": "array", "items": {"type": "string"}}},
            "required": ["colors"],
        },
    },
}]

response = client.chat.completions.create(
    model="gemma-4-31B-it",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Which colors appear in this image?"},
                {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_base64}"}},
            ],
        }
    ],
    tools=tools,
    tool_choice={"type": "function", "function": {"name": "report_colors"}},
    max_tokens=200,
    temperature=1.0,
)

print(response.choices[0].message.tool_calls[0].function.arguments)

cURL

curl -s "https://api.infercom.ai/v1/chat/completions" \
  -H "Authorization: Bearer $INFERCOM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma-4-31B-it",
    "messages": [{"role": "user", "content": [
      {"type": "text", "text": "Which colors appear in this image?"},
      {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,<BASE64>"}}
    ]}],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "colors",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {"colors": {"type": "array", "items": {"type": "string"}}},
          "required": ["colors"],
          "additionalProperties": false
        }
      }
    },
    "max_tokens": 200,
    "temperature": 1.0
  }'

See Function calling and JSON mode for the full tool-calling workflow.

Errors

Malformed image data

If the API cannot decode your image, it returns HTTP 400. This is a permanent error. Do not retry it. Check that the base64 string is complete and that the MIME type in the data: URL matches the file.

Caution

This response does not use the standard error envelope. The error field is a string, not an object, and there is no request_id. A client that reads error.message gets undefined here.

{
  "error": "Unable to decode image: cannot identify image file <_io.BytesIO object at 0x...>",
  "error_code": null,
  "error_param": null,
  "error_type": "invalid_request_error"
}

See API error codes for the standard envelope and the two paths that do not use it.

Image tokens count towards the context limit

Images consume prompt tokens. A large image plus a long prompt can exceed the model’s 128K context limit and return a context_length_exceeded error. Check usage.prompt_tokens in the response to see how many tokens an image costs.