Infercom Inference Service Models Overview and Specifications

The Infercom Inference Service provides access to a broad selection of AI models through the Global Model Catalog. Models are available in two categories: EU-hosted models running on Infercom’s sovereign infrastructure in Germany, and additional models available via global infrastructure.

EU-hosted models

The following models run on Infercom’s EU infrastructure in Germany. For these models, all data processing happens entirely within the EU — no data leaves EU jurisdiction, with full GDPR compliance and no US CLOUD Act exposure.

Text generation models

Developer	Model ID	Context length	Capabilities	Region	HF Link
OpenAI	gpt-oss-120b	128k tokens	Text, Reasoning	EU	Model card
Google	gemma-4-31B-it	128k tokens	Text, Vision	EU	Model card

Embedding model

Developer	Model ID	Context length	Capabilities	Region	HF Link
Mistral	E5-Mistral-7B-Instruct	4k tokens	Embeddings	EU	Model card

Audio model

Developer	Model ID	Max audio length	Capabilities	Region	HF Link
OpenAI	Whisper-Large-v3	25 MB	Transcription, Translation	EU	Model card

Global Model Catalog

In addition to EU-hosted models, the Infercom Inference Service provides access to a broader selection of models through the Global Model Catalog. Models not hosted in our EU datacenters are served via global infrastructure. You can always check a model’s hosting region via the API or the Playground.

Note

Every model is clearly labeled with its hosting region — both in the API response and in the Playground. You always know where your data is being processed.

Developer	Model ID	Context length	Region	Capabilities	HF Link
DeepSeek	DeepSeek-V3.1	128k tokens	Non-EU	Text, Reasoning	Model card
DeepSeek	DeepSeek-V3.2	32k tokens	Non-EU	Text, Reasoning	Model card
Meta	Meta-Llama-3.3-70B-Instruct	128k tokens	Non-EU	Text	Model card
MiniMax	MiniMax-M3	1M tokens	Non-EU	Text, Reasoning	Model card

Our provider lists MiniMax-M3 as a preview model: it has limited capacity and can be removed at short notice. Do not build production workloads on it. It accepts text only. Requests with image or video input return an error.

See Identifying model regions below for how to check where each model runs.

Recommended sampling parameters

Sampling parameters are model-specific. Use the values the model’s publisher recommends rather than a single setting across all models. The table below reproduces each publisher’s own recommendation, with a link to the source.

Model ID	temperature	top_p	top_k	Publisher recommendation
gpt-oss-120b	1.0	1.0	-	openai/gpt-oss
gemma-4-31B-it	1.0	0.95	64	Model card
MiniMax-M3	1.0	0.95	-	Model card
DeepSeek-V3.2	1.0	0.95	-	Model card
DeepSeek-V3.1	-	-	-	None published
Meta-Llama-3.3-70B-Instruct	-	-	-	None published

A dash means the publisher does not publish a value for that parameter. Where no recommendation exists, start at temperature 1.0 and tune for your own workload. DeepSeek-V3.1 publishes no prose guidance, but the generation config it ships defaults to temperature 0.6 and top_p 0.95.

The Infercom API accepts a temperature between 0 and 2. temperature=1.0 does not mean “maximum randomness” - it means the model’s probability distribution is used exactly as trained. Values below 1.0 sharpen it, values above 1.0 flatten it. Adjust temperature, top_p, or top_k, but generally not more than one of them at a time.

If you omit temperature

The Infercom API applies no service-wide default. What you get depends on the model.

Model ID	Behaviour when temperature is omitted
gemma-4-31B-it	Greedy decoding. Identical to temperature 0.
gpt-oss-120b	Greedy decoding. Identical to temperature 0.
DeepSeek-V3.2	Greedy decoding.
Meta-Llama-3.3-70B-Instruct	Greedy decoding.
MiniMax-M3	Greedy decoding.

Greedy decoding means the model always takes its highest-probability token. It is a valid choice for reproducible output. It is also the setting most likely to trigger the repetition loop described below.

/v1/responses and /v1/messages behave the same way. Set temperature on every request.

Caution

/v1/messages differs from the Anthropic API here. The Anthropic API documents a temperature default of 1.0. Infercom applies no default. Code that you port from the Anthropic SDK switches to greedy decoding unless you set temperature yourself.

Caution

Very low temperatures can cause non-terminating reasoning. Reasoning models can enter a repetition loop, restating the same point until max_tokens runs out, returning finish_reason: "length" and no usable answer. Raising max_tokens does not help. Omitting temperature counts as a very low temperature on most models, so this failure can appear in requests that set no sampling parameters at all. A low temperature is still a legitimate choice for reproducible or structured output - just move it back towards the recommended value if you see truncated responses. See Reasoning.

Defaults for the other sampling parameters

When you omit a sampling parameter, the platform applies the value below. These hold for every model.

Parameter	Applied when omitted	Effect
temperature	0.0	Any effective value below 0.001 is clipped, which forces greedy decoding
top_p	1.0	Unrestricted
top_k	1048576	Effectively unrestricted. This is a sentinel, not a vocabulary size - gemma-4-31B-it reports a vocabulary of 262,144
repetition_penalty	1.0	No penalty

Note

repetition_penalty is the only repetition control this API applies. frequency_penalty and presence_penalty are not implemented on any model, and any value other than 0 returns HTTP 400. See Rejected unless 0. Raise repetition_penalty only after you have set a correct temperature, and keep it at or below 1.2. The accepted range is 1 to 2, but values at or above 1.8 can stop a reasoning model from producing any content. It is honoured on gpt-oss-120b and gemma-4-31B-it, and ignored on DeepSeek-V3.1, DeepSeek-V3.2 and Meta-Llama-3.3-70B-Instruct. On MiniMax-M3 any value other than 1.0 returns HTTP 500. It applies on /v1/chat/completions only, and it penalises tokens in your prompt as well as generated ones.

Note

The byte-identical output you get when you omit temperature is not a pinned seed. It is the forced top_k of 1 described below. Do not rely on it for reproducibility across releases.

How top_k interacts with temperature

Caution

top_k has no effect unless temperature is 0.001 or above. Below that threshold, including when you omit temperature entirely, top_k is forced to 1 and decoding is greedy. Nothing in the response signals that your value was ignored.

To sample, set temperature explicitly.

The request below asks for sampling across 40 candidate tokens. It returns argmax instead, because temperature is absent:

{
  "model": "gemma-4-31B-it",
  "messages": [{"role": "user", "content": "Write one sentence about the sea."}],
  "top_k": 40
}

On gemma-4-31B-it, with an identical prompt:

Request	Result
top_k: 40, temperature omitted	Greedy. The same text on every request, top_k ignored
top_k: 40, temperature: 1.0	Sampling. The response varies on every request
No top_k, temperature omitted	Greedy. Byte-identical to the first row

Accepted range for top_k

Send top_k as 1 or greater. Unlike some other providers, -1 and 0 are not accepted as a way to disable it.

Value sent	/v1/chat/completions and /v1/messages	/v1/responses
-1	HTTP 400 Invalid value for 'top_k': -1 is less than the minimum of 1.	HTTP 200. Not validated
0	HTTP 400 Invalid value for 'top_k': 0 is less than the minimum of 1.	HTTP 200. Not validated
1	HTTP 200	HTTP 200
1048576	HTTP 200	HTTP 200
2000000	HTTP 200. No upper bound is enforced	HTTP 200

Caution

/v1/responses accepts -1 and 0 without an error, but does not treat them as “unrestricted”. Do not rely on that. Use a value of 1 or greater on every endpoint.

To leave top_k unrestricted, omit the parameter or send a value larger than the model’s vocabulary.

Identifying model regions

You can identify where a model is hosted through the API or the Playground.

Via the API

The /v1/models endpoint includes an sn_metadata object for each model. Use the region field to determine where a model is hosted: "EU" for sovereign models on Infercom’s EU infrastructure, or a non-EU region (e.g. "US", "JP") for models on global infrastructure.

Use the ?verbose=true query parameter to retrieve detailed model metadata including sovereignty information:

cURL

curl -s "https://api.infercom.ai/v1/models?verbose=true" \
  -H "Authorization: Bearer $INFERCOM_API_KEY" | \
  jq '.data[] | {id, region: .sn_metadata.region}'

Python

from openai import OpenAI
import requests

# Using requests for metadata inspection
headers = {"Authorization": f"Bearer {api_key}"}
response = requests.get(
    "https://api.infercom.ai/v1/models?verbose=true",
    headers=headers
)

for model in response.json()["data"]:
    region = model.get("sn_metadata", {}).get("region", "Global")
    print(f"{model['id']}: {region}")

Example response for an EU-hosted model:

{
  "id": "gpt-oss-120b",
  "object": "model",
  "sn_metadata": {
    "is_external": false,
    "region": "EU"
  }
}

Example response for a globally-routed model:

{
  "id": "DeepSeek-V3.1",
  "object": "model",
  "sn_metadata": {
    "is_external": true,
    "region": "JP"
  }
}

Via the Playground

In the Infercom Playground, region flags are displayed next to each model name, letting you see at a glance where each model runs.

Data sovereignty

Caution

EU sovereignty applies to EU-hosted models only. When using models from the Global Model Catalog that are not hosted on EU infrastructure, requests are processed on global infrastructure outside the EU. Always check the model’s region before processing sensitive or regulated data.

For EU-hosted models, Infercom provides:

• EU data residency — inference runs in our EU datacenters
• GDPR compliance — full compliance with EU data protection regulations
• No US CLOUD Act exposure — your inference data is not subject to US jurisdiction
• AI Act readiness — designed for compliance with the EU AI Act