> For the complete documentation index, see [llms.txt](https://docs.stackai.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.stackai.com/guides-and-quickstarts/evaluations-and-observability/create-an-evaluation-dataset.md).

# Create an evaluation dataset

An evaluation dataset contains inputs and the answers you expect the workflow to produce. Use it to check behavior consistently after a workflow change.

{% embed url="<https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FSQ1Sgy933krTyK0S4JpP%2Fuploads%2FR1fKiLkMkVbzYPlRuqhm%2FG081.mp4?alt=media&token=4dfec768-94a0-4ab0-9dc7-e5cc1b9b7621>" %}

## Before you start

Identify the project inputs and outputs with its builder before mapping fields. Choose the [evaluation method](/guides-and-quickstarts/evaluations-and-observability/configure-an-evaluator.md) so you know when a reference answer is needed. Reference-free evaluators can score cases without an expected answer; methods that compare against a reference require a reviewed one for each case.

## Choose the workflow fields

Review the workflow's required inputs and expected outputs before creating a dataset.

![Workflow input and output fields](https://content.gitbook.com/content/SQ1Sgy933krTyK0S4JpP/blobs/xN59iiLC1GAw7pRdZfP4/assets/g081-g081-01-echo.png)

Expected answers must represent the behavior you want to evaluate.

Open Evals → Datasets → New dataset, enter a name and description, and select the input and expected-output nodes.

![New dataset name and node mappings](https://content.gitbook.com/content/SQ1Sgy933krTyK0S4JpP/blobs/19QkFuXqBZkdFA8ZF4Uu/assets/g081-g081-02-create.png)

These mappings define the fields each dataset row must provide.

## Import existing runs

Review Import from runs before accepting captured outputs as expected answers.

![Import from runs controls](https://content.gitbook.com/content/SQ1Sgy933krTyK0S4JpP/blobs/f2opVbwThaFOeCrQoNIa/assets/g081-g081-03-import.png)

An earlier response is a useful starting point only if it is correct for the intended evaluation.

## Add and edit dataset rows

Choose Add row and enter an input with its correct expected output.

![Dataset row input and expected output](https://content.gitbook.com/content/SQ1Sgy933krTyK0S4JpP/blobs/Lw1qKRgGSvHwMc6giuBq/assets/g081-g081-04-alpha.png)

Keep the expected answer specific enough for the evaluator you plan to use.

Add more rows covering different requests the workflow should handle.

![Additional dataset case](https://content.gitbook.com/content/SQ1Sgy933krTyK0S4JpP/blobs/j1AMYeN9zOGJ89eV5sDa/assets/g081-g081-05-beta.png)

Consistent field mappings let the evaluator compare the resulting outputs across cases.

Include cases that help reveal incorrect behavior or evaluator weaknesses.

![Case intended to reveal incorrect behavior](https://content.gitbook.com/content/SQ1Sgy933krTyK0S4JpP/blobs/EqdJ6QEePpqBdADwLHkE/assets/g081-g081-06-gamma.png)

Review expected values carefully. An incorrect reference answer can cause a valid workflow response to fail evaluation.

## Import a CSV file

Open Import CSV to review the required headers and mapping instructions.

![CSV import and template controls](https://content.gitbook.com/content/SQ1Sgy933krTyK0S4JpP/blobs/oyKnpKyPDvRxkGEVgzqf/assets/g081-g081-07-csv.png)

Use Download template for the dataset's expected file structure before importing your rows.

## Review the saved dataset

Reopen a row to verify its saved input and expected output.

![Saved dataset row and Input & output mappings](https://content.gitbook.com/content/SQ1Sgy933krTyK0S4JpP/blobs/kloH8xHMUOuBgXyRG2db/assets/g081-g081-08-contract.png)

Check Input & output to confirm that the dataset still targets the intended nodes.

Review the complete dataset table and row sources.

![Dataset table and row sources](https://content.gitbook.com/content/SQ1Sgy933krTyK0S4JpP/blobs/mqsITN5ZNldk2Q0b30LY/assets/g081-g081-09-saved.png)

Correct mistaken reference values before using the dataset in an experiment.

## Important behavior

Review expected answers before using the dataset. A response imported from a past run is not automatically correct.

Include both typical inputs and inputs that are difficult for the workflow.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.stackai.com/guides-and-quickstarts/evaluations-and-observability/create-an-evaluation-dataset.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.