Implement Reasoning Models with Infercom

Reasoning models systematically process information to derive logical conclusions and solve complex problems step by step. Before producing a final answer, they generate an internal chain of thought - reasoning through the problem, exploring alternatives, and self-correcting. This makes them well suited for math, coding, multi-step planning, and analysis of ambiguous or complex inputs.

Reasoning models on Infercom

The following models support reasoning. Always check Supported models for current availability and regions.

Model ID	Region	How reasoning is controlled	Default
gpt-oss-120b	EU	reasoning_effort (low, medium, high)	medium
gemma-4-31B-it	EU	chat_template_kwargs: {"enable_thinking": true}	Off
DeepSeek-V3.1	Non-EU	Not configurable - always reasons	On
DeepSeek-V3.2	Non-EU	Not configurable - always reasons	On
MiniMax-M3	Non-EU	Not configurable - always reasons	On

The two controls are not interchangeable, and each applies to one model only:

• reasoning_effort adjusts how much gpt-oss-120b reasons. It cannot switch reasoning off.
• enable_thinking turns reasoning on or off for gemma-4-31B-it, which does not reason unless you ask it to.

Setting either parameter on a model that does not implement it is accepted and silently ignored - you get no error and no change in behavior.

How reasoning appears in the response

Reasoning-capable models return their chain of thought in a dedicated reasoning field on the message, separate from the final answer in content:

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "reasoning": "The user wants 17 * 23. 17 * 20 = 340, 17 * 3 = 51, 340 + 51 = 391.",
        "content": "17 * 23 = 391"
      },
      "finish_reason": "stop"
    }
  ]
}

Use reasoning for transparency or debugging, and content for the answer you show to users.

Note

For multi-turn conversations, only add the model’s final content back into the message history - do not feed previous reasoning back into the next turn.

Counting reasoning tokens

Every response reports the reasoning it produced in usage.completion_tokens_details.reasoning_tokens. Use this field to budget and to bill. It is populated on all EU models, including requests that carry an image.

{
  "usage": {
    "prompt_tokens": 21,
    "completion_tokens": 872,
    "completion_tokens_details": {
      "reasoning_tokens": 595
    },
    "total_tokens": 893
  }
}

For the prompt “What is 15 * 37? Explain.”:

Model	Request	reasoning_tokens
gpt-oss-120b	Default	39
gemma-4-31B-it	chat_template_kwargs: {"enable_thinking": true}	595
gemma-4-31B-it	Default, thinking off	0

reasoning_tokens counts towards completion_tokens, and therefore towards max_tokens. Subtract it from completion_tokens to see how many tokens went into the visible answer. MiniMax-M3 does not report reasoning_tokens.

Note

The reasoning_details array from the OpenAI Harmony format is not returned. Only the reasoning string and the reasoning_tokens count are available.

Budgeting tokens for reasoning

Caution

Reasoning tokens count toward your token limit. With reasoning models, the chain-of-thought the model generates before its final answer is counted against max_tokens (or max_completion_tokens). If you set this limit too low, the model can spend the entire budget thinking and get cut off before it produces a visible answer.

When that happens, the response comes back empty or truncated and finish_reason is "length". Always check finish_reason: if it is "length", raise max_tokens and retry. Budget generously for reasoning models - leave enough headroom for both the reasoning and the final answer.

This is the single most common source of “empty” or “broken” responses from reasoning models. Because the chain of thought is generated first and counts against your token limit, a low max_tokens value is consumed by reasoning before any answer is produced.

As a rule of thumb, allow several hundred to a few thousand tokens of headroom beyond the length of the answer you expect, depending on task complexity. For hard math, coding, or planning tasks, reasoning can be long - budget accordingly.

Caution

Low temperature can cause non-terminating reasoning. Reasoning models can enter a repetition loop, restating the same point in slightly different words until max_tokens runs out. You will see finish_reason: "length", a very high reasoning_tokens count, and no usable answer.

The lower the temperature, the more likely this is, and it is most pronounced at temperature=0. Raising max_tokens does not help here - the loop simply runs longer.

Sending no temperature at all puts you in exactly that state. The Infercom API applies no default, so most models decode greedily, which behaves like temperature=0. See If you omit temperature.

A low temperature remains a legitimate choice for reproducible or structured output. But if you use one and see truncated responses, raise the temperature towards the value the model publisher recommends before you raise max_tokens again. See Recommended sampling parameters.

Caution

A low temperature also silently disables top_k. If temperature is below 0.001, or absent, top_k is forced to 1 no matter what value you send. A request carrying top_k: 40 and no temperature returns argmax, and nothing in the response says so. Set temperature explicitly before you tune top_k. See How top_k interacts with temperature.

Controlling reasoning

Models with reasoning_effort

gpt-oss-120b accepts a reasoning_effort parameter (low, medium, high) that trades latency and token usage for answer quality. If you omit it, the model uses medium.

Note

Reasoning cannot be disabled on gpt-oss-120b. low is the minimum and still produces a short chain of thought. Values such as "none" or "off" are rejected with a 400. If you need a model that answers without reasoning, use gemma-4-31B-it, which has thinking off by default.

Python (OpenAI)

from openai import OpenAI

client = OpenAI(
    base_url="https://api.infercom.ai/v1",
    api_key="your-infercom-api-key",
)

response = client.chat.completions.create(
    model="gpt-oss-120b",
    messages=[{"role": "user", "content": "A bat and ball cost $1.10 total. The bat costs $1 more than the ball. How much is the ball?"}],
    reasoning_effort="high",
    max_tokens=2000,
)

print("Reasoning:", response.choices[0].message.reasoning)
print("Answer:", response.choices[0].message.content)

cURL

curl -s "https://api.infercom.ai/v1/chat/completions" \
  -H "Authorization: Bearer $INFERCOM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-oss-120b",
    "messages": [{"role": "user", "content": "A bat and ball cost $1.10 total. The bat costs $1 more than the ball. How much is the ball?"}],
    "reasoning_effort": "high",
    "max_tokens": 2000
  }'

Models with configurable thinking (enable_thinking)

gemma-4-31B-it has a configurable thinking mode that you turn on or off via chat_template_kwargs. Set enable_thinking to true to have the model reason before answering, or false for a direct answer.

Thinking is off by default. If you omit enable_thinking, the model answers directly and the reasoning field comes back empty - omitting it behaves exactly like setting it to false. You must pass true to get a chain of thought.

Caution

reasoning_effort has no effect on gemma-4-31B-it. The request succeeds, but the output is unchanged whether you send low, medium, or high. Use enable_thinking instead.

Python (OpenAI)

from openai import OpenAI

client = OpenAI(
    base_url="https://api.infercom.ai/v1",
    api_key="your-infercom-api-key",
)

response = client.chat.completions.create(
    model="gemma-4-31B-it",
    messages=[{"role": "user", "content": "A bat and ball cost $1.10 total. The bat costs $1 more than the ball. How much is the ball?"}],
    extra_body={"chat_template_kwargs": {"enable_thinking": True}},
    max_tokens=2000,
)

print("Reasoning:", response.choices[0].message.reasoning)
print("Answer:", response.choices[0].message.content)

cURL

curl -s "https://api.infercom.ai/v1/chat/completions" \
  -H "Authorization: Bearer $INFERCOM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma-4-31B-it",
    "messages": [{"role": "user", "content": "A bat and ball cost $1.10 total. The bat costs $1 more than the ball. How much is the ball?"}],
    "chat_template_kwargs": {"enable_thinking": true},
    "max_tokens": 2000
  }'

Note

With the OpenAI Python SDK, pass non-standard parameters like chat_template_kwargs inside extra_body, as shown above. When calling the API directly (cURL), include chat_template_kwargs at the top level of the request body.

Models that reason by default

MiniMax-M3 and the DeepSeek models reason by default - no extra parameter is required. Just send your request and read the reasoning and content fields from the response.

These models do not expose a control for how much they reason. They accept reasoning_effort for OpenAI compatibility, but it does not change their behavior. Budget max_tokens generously instead, as described above.

Troubleshooting: empty or truncated responses

If a reasoning model returns an empty content, a response that cuts off mid-sentence, or what looks like raw thinking in the output, the usual cause is an insufficient token budget - not a model defect. A second, less obvious cause is a temperature set well below the publisher’s recommended value, which can send the model into a repetition loop.

1. Check finish_reason

   Look at choices[0].finish_reason in the response. A value of "length" means the model hit your max_tokens limit before it finished. "stop" means it completed normally.

2. Raise max_tokens

   If finish_reason is "length", increase max_tokens (or max_completion_tokens) and retry. Give the model enough room for both its reasoning and the final answer.

3. Check your temperature if raising max_tokens does not help

   Compare usage.completion_tokens_details.reasoning_tokens with your max_tokens. If the reasoning consumes the entire budget again at a higher limit, you are likely in a repetition loop. Set temperature to the value the publisher recommends for that model. Check that your request sends temperature at all - an absent parameter means greedy decoding, not a mid-range default.

4. Verify the reasoning and content fields

   Confirm that reasoning contains the chain of thought and content contains the final answer. With an adequate budget and a sensible temperature, the two are cleanly separated.

Common failure signatures:

Symptom	What you see	Fix
Empty answer	content is null or empty, finish_reason is "length"	Raise max_tokens
Truncated answer	content cuts off mid-sentence, finish_reason is "length"	Raise max_tokens
Reasoning leaks into content	Chain of thought appears in content because the model was cut off before it emitted the end-of-thinking marker. When this happens the reasoning field is empty even though usage.completion_tokens_details.reasoning_tokens is non-zero	Raise max_tokens
Reasoning never terminates	finish_reason is "length" and reasoning_tokens consumes the whole budget at every limit you try, with the chain of thought restating the same point	Raise the temperature towards the publisher’s recommended value

Prompting reasoning models

• Keep system prompts minimal. Excessive instructions can constrain the model’s reasoning scope and reduce quality.
• Avoid adding your own “think step by step” chain-of-thought prompting - the model already reasons internally. Use clear, zero-shot or single-instruction prompts.
• Always send temperature explicitly. The Infercom API has no default. Omit the parameter and most models decode greedily, which is the setting most likely to trigger the repetition loop above. It also forces top_k to 1, so any top_k you send is ignored.
• Use the sampling parameters the model publisher recommends. They differ per model and are listed under Recommended sampling parameters. For gpt-oss-120b the publisher recommends temperature 1.0 and top_p 1.0.
• temperature=1.0 does not mean “maximum randomness”. It means the model’s probability distribution is used exactly as trained. Values below 1.0 sharpen it, values above 1.0 flatten it. The Infercom API accepts values from 0 to 2.

Use cases

Report generation

Reasoning models are well suited to processing unstructured information - legal contracts, financial statements, or scientific papers - recognizing patterns across multiple facets of the input before distilling it into a comprehensive summary.

Planning for workflows and agents

Reasoning models excel at ambiguous, complex tasks. They can break down intricate problems, strategize solutions, and make decisions over large volumes of uncertain information - making them effective planners and orchestrators in agentic systems.

Coding and mathematics

These models are effective at reviewing and improving code, catching subtle issues a quick read might miss, and breaking down math problems into verifiable steps.

Best practices

Budget tokens for reasoning

Reasoning counts toward max_tokens. Set the limit high enough for both the chain of thought and the final answer. If responses come back empty or truncated, check finish_reason - a value of "length" means you need to raise max_tokens.

Latency and cost

Reasoning outputs have higher latency and token usage because of the chain-of-thought process. For simpler tasks, consider a non-reasoning model (or disable thinking where the model supports it) to optimize for budget and response time.

Streaming

Enabling streaming improves perceived responsiveness, since reasoning models take longer to produce a complete answer. Add stream=True to your request to display tokens as they become available.

FAQs

Why do I get an empty or truncated response with finish_reason: length?

There are two common causes. First, reasoning tokens count against max_tokens, so the budget can be consumed before any visible answer is produced - raise max_tokens. Second, at a low temperature the model can enter a repetition loop that never terminates - raising max_tokens will not help, and you should move the temperature towards the value the publisher recommends for that model.

To tell them apart, check usage.completion_tokens_details.reasoning_tokens. If it exactly equals your max_tokens no matter how high you set the limit, suspect a loop.

Which temperature and top_p should I use?

Use the values the model publisher recommends - they differ per model and are listed under Recommended sampling parameters. There is no single value that suits every model. The Infercom API accepts a temperature from 0 to 2, and temperature=1.0 means the model’s distribution is used exactly as trained rather than “maximum randomness”.

What is the difference between reasoning_effort and enable_thinking?

They are two different controls on two different models, and neither works on the other’s model. gpt-oss-120b uses reasoning_effort (low/medium/high) to dial how much it reasons; it defaults to medium and cannot be turned off. gemma-4-31B-it uses chat_template_kwargs: {"enable_thinking": true/false} to turn reasoning on or off; it is off by default. Setting either parameter on a model that does not implement it is accepted and silently ignored, so check the table above before relying on one.

How do I turn reasoning off completely?

Only gemma-4-31B-it can answer without reasoning - and it already does so by default, since enable_thinking is off unless you set it. Reasoning cannot be disabled on gpt-oss-120b, MiniMax-M3 or the DeepSeek models; reasoning_effort: "none" is not a valid value and returns a 400. The lowest setting available anywhere is reasoning_effort: "low" on gpt-oss-120b, which still produces a short chain of thought. On MiniMax-M3, enable_thinking: false does not switch reasoning off: the reasoning then appears inside content, wrapped in <mm:think> tags.

I got a 400 error about maximum context length. Where can I check the limit?

See the Supported models page for each model’s context length.

Where are these models hosted?

Models marked EU run in Infercom’s EU data centers in Germany with full data sovereignty. Check the Supported models page to verify each model’s region.