Vision and Multimodal Guide - Infercom Documentation

Infercom provides access to vision-capable models on our EU sovereign infrastructure, allowing you to process both text and images with full data sovereignty. These models analyze images and generate context-aware text responses.

Supported models

Model	Context	Region	Notes
`gemma-4-31B-it`	128K	EU	Google’s Gemma 4 31B with vision capabilities

gemma-4-31B-it runs on Infercom’s sovereign infrastructure in Germany. Your image data never leaves the EU.

Make a query with an image

On Infercom, the vision model request follows OpenAI’s multimodal input format which accepts both text and image inputs in a structured payload. While the call is similar to Text Generation, it differs by including an encoded image file, referenced via the image_path variable. A helper function is used to convert this image into a base64 string, allowing it to be passed alongside the text in the request.

Step 1

Make a new Python file and copy the code below.

This example uses gemma-4-31B-it, Google’s vision-capable Gemma 4 model hosted on Infercom’s EU sovereign infrastructure.

from sambanova import SambaNova
import base64

client = SambaNova(
    base_url="https://api.infercom.ai/v1",
    api_key="your-infercom-api-key",
)

# Helper function to encode the image
def encode_image(image_path):
  with open(image_path, "rb") as image_file:
    return base64.b64encode(image_file.read()).decode('utf-8')

# The path to your image
image_path = "sample.JPEG"

# The base64 string of the image
image_base64 = encode_image(image_path)

print(image_base64)

response = client.chat.completions.create(
    model="gemma-4-31B-it",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What is happening in this image?"},
                {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_base64}"}}
            ]
        }
    ]
)

print(response.choices[0].message.content)

from openai import OpenAI
import base64

client = OpenAI(
    base_url="https://api.infercom.ai/v1",
    api_key="your-infercom-api-key",
)

# Helper function to encode the image
def encode_image(image_path):
  with open(image_path, "rb") as image_file:
    return base64.b64encode(image_file.read()).decode('utf-8')

# The path to your image
image_path = "sample.JPEG"

# The base64 string of the image
image_base64 = encode_image(image_path)

print(image_base64)

response = client.chat.completions.create(
    model="gemma-4-31B-it",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What is happening in this image?"},
                {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_base64}"}}
            ]
        }
    ]
)

print(response.choices[0].message.content)

Step 2

Use your Infercom API key from the API keys and URLs page to replace the placeholder "your-infercom-api-key" in the construction of the client.

Step 3

Select an image and move it to a suitable path that you can specify in the lines.

# The path to your image
image_path = "sample.JPEG"

Step 4

Verify the prompt to pair with the image in the content portion of the user prompt.

Step 5

Run the Python file to receive the text output.

Tools and structured output with images

Vision requests support the same tools, tool_choice and response_format options as text-only requests. The constraint is applied during generation, so a forced tool call or a strict JSON schema is honoured on a request that carries an image.

Option	Behaviour on a request with an image
`tool_choice` with a function name	The model returns that tool call. `finish_reason` is `tool_calls`
`tool_choice: "none"`	The model returns plain text. `finish_reason` is `stop`
`response_format: {"type": "json_schema", ...}`	The model returns JSON that matches the schema

from openai import OpenAI
import base64

client = OpenAI(
    base_url="https://api.infercom.ai/v1",
    api_key="your-infercom-api-key",
)

with open("sample.JPEG", "rb") as image_file:
    image_base64 = base64.b64encode(image_file.read()).decode("utf-8")

tools = [{
    "type": "function",
    "function": {
        "name": "report_colors",
        "parameters": {
            "type": "object",
            "properties": {"colors": {"type": "array", "items": {"type": "string"}}},
            "required": ["colors"],
        },
    },
}]

response = client.chat.completions.create(
    model="gemma-4-31B-it",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Which colors appear in this image?"},
                {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_base64}"}},
            ],
        }
    ],
    tools=tools,
    tool_choice={"type": "function", "function": {"name": "report_colors"}},
    max_tokens=200,
    temperature=1.0,
)

print(response.choices[0].message.tool_calls[0].function.arguments)

curl -s "https://api.infercom.ai/v1/chat/completions" \
  -H "Authorization: Bearer $INFERCOM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma-4-31B-it",
    "messages": [{"role": "user", "content": [
      {"type": "text", "text": "Which colors appear in this image?"},
      {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,<BASE64>"}}
    ]}],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "colors",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {"colors": {"type": "array", "items": {"type": "string"}}},
          "required": ["colors"],
          "additionalProperties": false
        }
      }
    },
    "max_tokens": 200,
    "temperature": 1.0
  }'

See Function calling and JSON mode for the full tool-calling workflow.

Errors

Malformed image data

If the API cannot decode your image, it returns HTTP 400. This is a permanent error. Do not retry it. Check that the base64 string is complete and that the MIME type in the data: URL matches the file.

This response does not use the standard error envelope. The error field is a string, not an object, and there is no request_id. A client that reads error.message gets undefined here.

{
  "error": "Unable to decode image: cannot identify image file <_io.BytesIO object at 0x...>",
  "error_code": null,
  "error_param": null,
  "error_type": "invalid_request_error"
}

See API error codes for the standard envelope and the two paths that do not use it.

Image tokens count towards the context limit

Images consume prompt tokens. A large image plus a long prompt can exceed the model’s 128K context limit and return a context_length_exceeded error. Check usage.prompt_tokens in the response to see how many tokens an image costs.

Get started

Models

Features

Agentic Coding

Build

Resources

Implement Vision and Multimodal Features - Developer Guide

Supported models

Make a query with an image

Tools and structured output with images

Errors

Malformed image data

Image tokens count towards the context limit

​Supported models

​Make a query with an image

​Tools and structured output with images

​Errors

​Malformed image data

​Image tokens count towards the context limit

Supported models

Make a query with an image

Tools and structured output with images

Errors

Malformed image data

Image tokens count towards the context limit