> For the complete documentation index, see [llms.txt](https://docs.stackai.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.stackai.com/technical-reference/files-knowledge-and-retrieval/chunking-embeddings-and-indexing.md).

# Chunking, Embeddings, and Indexing

Indexing prepares parsed content for retrieval. It divides content into chunks, creates embeddings, and writes searchable records. These stages occur after upload or source synchronization.

{% embed url="<https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FSQ1Sgy933krTyK0S4JpP%2Fuploads%2Fm3Ojdz3X7SNQyjyP9kZL%2FG039.mp4?alt=media&token=4b4833ff-28d9-4cfb-9fa9-8d1d4a3f7fdf>" %}

## Before you begin

An accessible knowledge base and source files that can be parsed.

## How indexing works

| Term      | Definition                                                                                                                                    |
| --------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| Chunk     | A piece of source content stored or passed as a retrieval unit. Its boundaries affect how much surrounding evidence appears in a result.      |
| Embedding | A numerical representation used for similarity-based search. It supports semantic matching; it is not the source document or a citation.      |
| Index     | Searchable records prepared from source content. A file must complete required processing before the index can return its content.            |
| Parsing   | Extraction of usable text or structure from source content. Review the extracted content when the workflow misses text, tables, or locations. |

## Configuration options

| Option        | Purpose                                 | Behavior                                                                     |
| ------------- | --------------------------------------- | ---------------------------------------------------------------------------- |
| Chunk overlap | Content shared between adjacent chunks. | It can preserve continuity while also creating repeated passages in results. |

## Configure document indexing

1. Review the knowledge base’s chunking, OCR, and embedding settings.
2. Choose supported processing settings before ingestion. If the interface permits a later change, follow its reprocessing instructions.

![Open Update Settings before ingestion and review the available processing controls.](https://content.gitbook.com/content/SQ1Sgy933krTyK0S4JpP/blobs/5fgOG9YDCIEi9xbGDVIY/assets/g039-g039-01-settings.png)

Open Update Settings before ingestion and review the available processing controls.

3. Wait for those files to reach a usable indexed state, then inspect retrieval results.

Search returns relevant pieces of the indexed source material.

## Behavior and limitations

Chunk size affects how much context appears in each result. Small chunks can lose surrounding context; large chunks can add irrelevant text. Some processing settings become read-only after ingestion. Review them before uploading. If reprocessing is available, confirm completion before expecting retrieval to reflect the new settings.

## Related resources

[Chunking, OCR, and embedding settings](https://docs.stackai.com/guides-and-quickstarts/knowledge-and-retrieval/chunking-ocr-and-embedding-settings.md)

[File Inputs, Supported Formats, and Size Limits](https://docs.stackai.com/technical-reference/files-knowledge-and-retrieval/file-inputs-supported-formats-and-size-limits.md)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.stackai.com/technical-reference/files-knowledge-and-retrieval/chunking-embeddings-and-indexing.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
