> For the complete documentation index, see [llms.txt](https://docs.stackai.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.stackai.com/guides-and-quickstarts/workflow-builder/configure-models-and-prompts/choosing-thinking-levels.md).

# Choosing thinking levels

Choose thinking settings for StackAI workflow agents, balance quality with response time and cost, and understand provider differences.

Choose thinking for the work an AI Agent node must do. Use less for explicit extraction and straightforward transformations. Use more when the agent must connect evidence, interpret exceptions, or decide what to do next. Keep the lowest setting that reliably meets your workflow's quality requirements.

This applies across providers, including models that offer an on/off switch, adjustable effort, a token budget, or reasoning built into the model. The available controls depend on the selected model and its StackAI connection.

## What thinking changes

Thinking, also called reasoning, lets a model spend additional computation working through a problem. Higher effort can improve difficult decisions, but usually adds latency and generated tokens. It does not guarantee correctness, and the final answer can still be short. AWS describes these tradeoffs in its [model reasoning guide](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-reasoning.html).

Model choice and thinking level are separate decisions. Compare models and their supported settings on the same inputs. A stronger model at Low can outperform another model at High; the labels do not establish a quality ranking across models. [OpenAI model selection](https://developers.openai.com/api/docs/guides/model-selection)

## Choose a starting level

Use these workflow examples as starting points for testing. Each level applies only where the selected model supports it.

| Setting                         | Work inside the node                                                                          | StackAI examples                                                                                                              |
| ------------------------------- | --------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| Off or lowest supported setting | The required facts are explicit and the task follows a clear procedure.                       | Extract labeled fields, translate text, format an approved response, or answer a direct question from one retrieved passage.  |
| Low                             | A narrow task needs some interpretation and has clear rules.                                  | Classify support tickets, summarize a straightforward call, or extract fields from documents with varied layouts.             |
| Medium                          | The agent combines information, applies several rules, or makes a bounded plan.               | Draft an answer using a customer record and several knowledge-base passages, or apply a policy with a few exceptions.         |
| High                            | Dependent reasoning steps, conflicting evidence, or complex exceptions determine the result.  | Reconcile an invoice against an order and receipt, interpret a policy amendment, or investigate a discrepancy across systems. |
| Extra high or Max               | A demanding task still fails at a lower level and testing shows that additional effort helps. | Difficult cross-document analysis or a complex investigation that can tolerate a longer response time.                        |

These categories adapt the providers' general guidance to workflow nodes. Google distinguishes simple retrieval and classification from comparisons and complex planning; OpenAI highlights ambiguity and relationships across documents as good reasoning use cases. [Google thinking guidance](https://ai.google.dev/gemini-api/docs/thinking), [OpenAI reasoning best practices](https://developers.openai.com/api/docs/guides/reasoning-best-practices)

Increase thinking when the agent has the right information but repeatedly draws the wrong conclusion, misses an exception, or chooses an unsuccessful sequence of actions. A consequential decision deserves a stricter evaluation threshold, but consequence alone does not prove that maximum effort improves the result.

## Set thinking per agent node

Workflow length is a weak signal. Ten nodes can perform ten straightforward tasks. One AI Agent node can make several dependent searches, compare the results, and decide whether it has enough evidence.

For an invoice workflow, a useful starting configuration is:

| Step                                   | Starting configuration | Reason                                 |
| -------------------------------------- | ---------------------- | -------------------------------------- |
| Extract invoice fields                 | Low                    | Identify values and their labels.      |
| Fetch the purchase order               | App action             | Retrieve the specified record.         |
| Compare totals and tolerances          | Code or explicit rules | Apply a defined calculation.           |
| Interpret discrepancies and exceptions | Medium or High         | Combine the evidence and apply policy. |
| Format the result                      | Off or Low             | Present an already determined outcome. |

For a support assistant, a direct FAQ answer may work at Off or Low. A request involving conflicting policy passages and a customer-specific exception is a stronger candidate for Medium or High. Both use retrieval; their reasoning demands differ.

Tool access alone is also a weak signal. Looking up a known record is straightforward. Deciding which records to search, adapting after an unexpected result, and resolving contradictory information require more judgment.

Use explicit workflow steps when the procedure is known. Splitting a task into fixed subtasks and checking intermediate results can make each model call easier. [Anthropic Building effective agents](https://www.anthropic.com/engineering/building-effective-agents)

## Find the controls in StackAI

Select the provider and model, then review the Thinking or Reasoning controls shown for that AI Agent node. Some models have an effort selector; others have a toggle or rely on their default behavior. Use the available values for that specific model.

**Off does not always mean zero reasoning.** A model that requires reasoning can continue at its lowest supported effort. Some Claude configurations retain low adaptive thinking for tool use. To eliminate optional reasoning overhead, choose a model and connection that support disabling it, then verify the resulting behavior.

Provider API capabilities can be broader than the controls exposed in StackAI. The provider notes below explain those differences; they do not imply that every API parameter is available in the node.

## Provider differences

Provider details reviewed on October 12, 2026. Recheck supported values and defaults when changing a model, deployment, or connection.

### OpenAI

OpenAI models can offer different subsets of None, Minimal, Low, Medium, High, Extra high, and Max. Defaults also vary. For example, GPT-5 supports Low, Medium, and High, while GPT-6 Astra cannot disable reasoning. A provider-wide recommendation such as “always use Medium” will not fit every model.

In StackAI, supported OpenAI models expose Reasoning settings. Off resolves to the lowest supported option, which may still involve reasoning. Start with a supported lower setting for routine work and compare higher settings for difficult decisions. [OpenAI reasoning models](https://developers.openai.com/api/docs/guides/reasoning)

### Anthropic Claude

Claude models can use a manual thinking budget or adaptive thinking, depending on the model and API. With adaptive thinking, the model decides how much to think, while effort influences how much work it performs. Effort can also affect tool use and self-checking. It is not a fixed token allocation.

In StackAI, supported Claude models expose Thinking as a toggle or effort selector. Higher levels are useful candidates for demanding analysis; use lower levels when evaluations show quality holds. Some models always think. Changing top-level effort within a cached conversation can also affect cache reuse, so distinguish tuning separate workflow nodes from changing settings repeatedly inside one conversation. [Claude effort](https://platform.claude.com/docs/en/build-with-claude/effort), [Claude extended thinking](https://platform.claude.com/docs/en/build-with-claude/extended-thinking)

### Google Gemini

Gemini controls depend on the model and API. The Generate Content API uses thinking levels for Gemini 3 models and token budgets for Gemini 2.5 models. Some models support disabling thinking; others do not. Minimal is not a universal off switch.

StackAI's standard Google agent configuration currently leaves thinking configuration to the model and SDK defaults, rather than exposing a Gemini thinking-level selector. When no selector appears, compare available models and workflow designs. Do not assume that an API example for a thinking budget can be entered into the node. [Gemini Generate Content thinking](https://ai.google.dev/gemini-api/docs/generate-content/thinking)

### Microsoft Azure

For Azure-hosted OpenAI models, use the deployed model's reasoning capabilities as the starting point. Accepted settings and defaults vary by model; deployment and API support matter too.

The standard StackAI Azure provider currently has no dedicated reasoning picker. Confirm the connection's supported controls and test the deployed model's behavior instead of assuming parity with the OpenAI provider's UI. [Azure OpenAI reasoning models](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/reasoning)

### Amazon Bedrock

Bedrock hosts models from several vendors, so its reasoning behavior follows the chosen model. Claude on Bedrock can use extended or adaptive thinking, with different configuration requirements across versions. Other Bedrock models need their own capability checks.

StackAI exposes Thinking for supported Bedrock Claude models. Support can differ from a direct Anthropic connection; an always-on model or a tool-use configuration may retain reasoning when Off is selected. Compare the exact model and connection you plan to run. [Bedrock Claude thinking](https://docs.aws.amazon.com/bedrock/latest/userguide/claude-messages-extended-thinking.html)

### Together AI

Together distinguishes models that always reason, hybrid models that can switch reasoning on or off, and models with adjustable effort. Supported effort values can differ, and some values may be mapped to another level. A shared API parameter name does not establish identical behavior across hosted models.

StackAI currently exposes an effort selector for its supported Together AI Kimi K3 configuration, with Low, High, and Max. Other models in the catalog may rely on defaults without a selector. Use the options shown for the selected model. [Together reasoning](https://docs.together.ai/docs/inference/chat/reasoning)

### xAI Grok

Grok reasoning controls vary by model. Grok 4.5 and 4.6 offer adjustable effort and cannot disable reasoning. A further distinction applies to Grok 4.20 Multi-Agent: its effort setting controls the number of collaborating agents, rather than ordinary reasoning depth.

StackAI's standard xAI provider currently has no effort picker. Choose the available model appropriate to the task and test its default behavior. A multi-agent model is not equivalent to setting a single agent node to High. [xAI reasoning](https://docs.x.ai/developers/model-capabilities/text/reasoning)

### Mistral

Mistral's reasoning options are model-specific. Its documented hybrid models can use `reasoning_effort` values such as None and High; None can mean minimal thinking with no thinking chunk in the response. It does not supply a universal Low, Medium, High scale.

The standard StackAI Mistral provider currently has no reasoning picker. Compare the available model variants and their behavior. Do not use the presence or absence of a visible thinking trace as proof of reasoning depth. [Mistral reasoning](https://docs.mistral.ai/studio/conversations/reasoning)

### Perplexity

Perplexity combines reasoning with research and search behavior. Its Agent API presets configure more than thinking alone, including the model and tools. Comparing a quick lookup with deep research therefore changes more than one parameter.

StackAI's current Perplexity catalog uses Sonar model selections without a thinking-level picker. Perplexity now documents a transition from Sonar requests to Agent API presets, so verify endpoint behavior when evaluating these models. A provider research preset should not be treated as a StackAI node effort setting. [Perplexity Sonar migration](https://docs.perplexity.ai/docs/agent-api/migrate-from-sonar/overview)

### Other model providers and hosting routes

For the following StackAI provider choices, the standard node currently has no dedicated thinking selector. Apply the same task-based heuristics by choosing the model and testing its actual behavior. Hosting speed, an OpenAI-compatible endpoint, and a model name containing “thinking” do not establish support for adjustable effort.

| StackAI provider | How to apply the guidance                                                                                                                             |
| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| Meta             | Compare the available Llama models on the task; a Llama selection does not imply an effort control.                                                   |
| Groq             | Check the exact model. An instruct model and a reasoning model such as GPT-OSS can behave differently even on the same host.                          |
| Cerebras         | Choose among the available models and measure complete workflow latency and quality.                                                                  |
| Fireworks AI     | Check the selected hosted model's default reasoning behavior; the catalog includes different model families.                                          |
| Baseten          | Evaluate the selected model and deployment together. Do not assume another host's settings carry over.                                                |
| Modular          | Check the selected model and endpoint configuration; model-family behavior determines the relevant comparison.                                        |
| Core42           | Distinguish the chosen model family and its endpoint defaults from controls offered by its original vendor.                                           |
| Replicate        | Compare the selected model variant and its available endpoint inputs.                                                                                 |
| Local            | Confirm what the configured endpoint accepts and what StackAI sends. OpenAI compatibility does not guarantee support for the same reasoning settings. |

Speech recognition, including Deepgram transcription, is a separate task. Choose its transcription model for recognition quality and latency, then configure thinking on the downstream language-model agent that interprets the transcript.

## Diagnose failures before increasing thinking

| Observed problem                                                               | First adjustment                                                                                    |
| ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------- |
| The necessary source is absent, stale, or unreadable                           | Fix retrieval, synchronization, access, parsing, or OCR.                                            |
| Relevant evidence is present, but the agent misses a relationship or exception | Compare a higher thinking level.                                                                    |
| Required fields or output structure are invalid                                | Clarify instructions and apply structured output validation.                                        |
| The agent chooses an unsuccessful tool sequence                                | Improve tool descriptions and task constraints, then compare higher effort if planning still fails. |
| The answer is incomplete                                                       | Inspect the finish reason, token limit, and execution limits.                                       |
| The output is too verbose                                                      | Specify the desired answer length and format.                                                       |

More reasoning cannot supply a document that the agent never received. Likewise, reducing the output limit is not a reliable substitute for lowering thinking effort. On providers where thinking and answer tokens share a limit, a tight cap can stop generation before the answer is complete. [Google token limits](https://ai.google.dev/gemini-api/docs/thinking#token-limits-and-max_output_tokens)

## Validate the setting on real workflow cases

1. Define the required outcome: correct fields, a supported conclusion, appropriate sources, and successful actions.
2. Build a representative evaluation dataset with ordinary cases, exceptions, conflicting sources, and missing information.
3. Compare adjacent supported levels while keeping the model, prompt, retrieval settings, and inputs fixed. If effort is unavailable, compare model variants as a separate experiment.
4. Measure node quality and the final workflow result. Include response time, total token usage or cost, tool calls, and retries.
5. Repeat variable cases and keep the lightest configuration that meets the quality threshold. Re-evaluate after changing the model, prompt, tools, or retrieved context.

Measure cost per successful workflow, including retries, rather than assuming the cheapest individual call produces the cheapest successful result. An interactive assistant also needs a response-time target; a scheduled batch job may tolerate more thinking.

The providers' recommendations are starting points. The best configuration is the one that passes your own cases at an acceptable cost and response time. [OpenAI model selection](https://developers.openai.com/api/docs/guides/model-selection), [Claude effort guidance](https://platform.claude.com/docs/en/build-with-claude/effort)

## Related guides

* [Configure models and prompts](https://docs.stackai.com/guides-and-quickstarts/workflow-builder/configure-models-and-prompts)
* [Model Settings and Supported Capabilities](https://docs.stackai.com/technical-reference/models-agents-and-skills/model-settings-and-supported-capabilities)
* [Create an evaluation dataset](https://docs.stackai.com/guides-and-quickstarts/evaluations-and-observability/create-an-evaluation-dataset)
* [Run and inspect an experiment](https://docs.stackai.com/guides-and-quickstarts/evaluations-and-observability/run-and-inspect-an-experiment)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.stackai.com/guides-and-quickstarts/workflow-builder/configure-models-and-prompts/choosing-thinking-levels.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
