Advanced Settings

The Advanced Settings give you fine-grained control over how the large language model (LLM) behaves, how it handles errors, and how it protects sensitive data.

Stream Data

Default: On

When this is on, the AI's response appears word by word as it is being written. When it is off, the workflow waits for the entire response to be finished before passing it along.

Note: Some Claude models require Stream Data turned on for long responses (e.g. writing extensive documents) that takes a few minutes or more to complete.

Safe Context Window

Default: Off

Safe context window automatically reduces the response to the model's maximum context size.

This setting is turned off by default. When the setting is off, you will get an error message if your context gets too big.

Note: Turning this on means you might unexpectedly lose meaningful context.

File Access

Default: Off

When you pass very large pieces of content to the LLM (e.g. the content of a large PDF file), loading it all directly in the prompt can be wasteful and slow.

With File Access turned on, large inputs are saved as files in a secure sandbox, and the AI is given a tool to read from them on demand instead of having everything crammed into the prompt.

Data & Time

Default: On

When turned on, the LLM will be aware of the current date & time.

Guardrails

Default: Off

Guardrails screen the LLM response for harmful or inappropriate content. If a violation is detected, the workflow can surface a warning or block the content entirely.

You can choose which types of content to screen for:

• Toxic Content: Catches hateful, abusive, or offensive language.

• Legal Advice: Flags responses that provide unlicensed legal guidance.

• Suicidal Thoughts: Detects content involving self-harm or suicidal ideation.

PII Compliance

Default: Off

PII stands for Personally Identifiable Information. When this setting is on, the node scans for PII in the input before sending it to the LLM.

For each type of PII, you can choose how to handle it:

• Warning: The PII is detected and flagged in logs, but the original value is still sent to the model.

• Encrypt: The PII is replaced with a coded placeholder before being sent to the model. The original value is restored in the output, so the AI never sees the raw sensitive data.

The four categories you can configure are: Email, Phone, SSN (Social Security Number), and Credit Card.

Temperature

Default: 0 (most focused) Range: 0 to 1

Increase temperature in order to increase randomness in the output, making the model less deterministic.

Max Output Length

Default - 3000.

Default: 10,000 tokens (~7,500 words) Range: 0 to the model's maximum

This sets the maximum length of the LLM response. A "token" is roughly ¾ of a word in English. Increasing this slider will allow the model to give more verbose answers.

Retry on Failure

Default: Off

Sometimes a LLM model fails temporarily due to high traffic, a brief provider outage, or a network hiccup. You can reduce workflow failure from these temporary issues with retry options:

• Max Retries: How many times to retry before giving up. You can set between 1 and 10 retries. Default when enabled is 2.

• Retry Interval: How long to wait between each retry attempt, in milliseconds. The default is 1,000 ms (1 second). You can set anything from 100 ms to 20 seconds.

LLM Fallback Mode

Default: Off

A fallback model is a backup LLM that automatically takes over if the primary model fails, even after all retries. You choose which provider and model acts as the backup.

When you turn this on, you can specificy:

• Fallback Provider: The AI company whose model will be used as the backup (e.g. Anthropic, OpenAI, Google).

• Fallback Model: The specific model from that provider to fall back to.

Fallback Branch

Default: Off

When turned ON, you can specify a different flow to follow in case this LLM node's execution fails, such as sending a notification message or go down a different path.