Skip to content

[docs] Add more guidance on products metadata #1836

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

[docs] Add more guidance on products metadata #1836

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2319478
add more guidance on products
colleenmcginnis Sep 5, 2025
8f210ab
apply suggestions from code review
colleenmcginnis Sep 9, 2025
039d581
update docs/syntax/frontmatter.md
colleenmcginnis Sep 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions docs/_snippets/products-list.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
`products` is a list of objects, each object has an `id` field.

```yaml
products:
- id: apm
```
Each product ID is mapped to a full product name.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where's the mapping defined? Shall we point to the source code?

Copy link
Contributor

@cotti cotti Sep 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@theletterf At this moment on main, it's in the code - I'm working on a change that will migrate this to a separate file. I'll update the documentation accordingly by the time the PR is up.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I was just expecting the reader to look at the table in the dropdown. Maybe we should make that clearer?

You use the product ID in the source files, and docs-builder will use the full product name in
a `product_name` `meta` tag in the generated HTML and YAML metadata in generated Markdown files,
which are used to drive elastic.co search functionality.

:::{dropdown} Valid products IDs

| Product ID | Product name |
|---------------------------------------------|-----------------------------------------------|
| `apm` | APM |
| `apm-agent` | APM Agent |
| `auditbeat` | Auditbeat |
| `beats` | Beats |
| `cloud-control-ecctl` | Elastic Cloud Control ECCTL |
| `cloud-enterprise` | Elastic Cloud Enterprise |
| `cloud-hosted` | Elastic Cloud Hosted |
| `cloud-kubernetes` | Elastic Cloud Kubernetes |
| `cloud-serverless` | Elastic Cloud Serverless |
| `cloud-terraform` | Elastic Cloud Terraform |
| `ecs` | Elastic Common Schema (ECS) |
| `ecs-logging` | ECS Logging |
| `edot-cf` | EDOT Cloud Forwarder |
| `edot-sdk` | Elastic Distribution of OpenTelemetry SDK |
| `edot-collector` | Elastic Distribution of OpenTelemetry Collector |
| `elastic-agent` | Elastic Agent |
| `elastic-serverless-forwarder` | Elastic Serverless Forwarder |
| `elastic-stack` | Elastic Stack |
| `elasticsearch` | Elasticsearch |
| `elasticsearch-client` | Elasticsearch Client |
| `filebeat` | Filebeat |
| `fleet` | Fleet |
| `heartbeat` | Heartbeat |
| `integrations` | Integrations |
| `kibana` | Kibana |
| `logstash` | Logstash |
| `machine-learning` | Machine Learning |
| `metricbeat` | Metricbeat |
| `observability` | Elastic Observability |
| `packetbeat` | Packetbeat |
| `painless` | Painless |
| `search-ui` | Search UI |
| `security` | Elastic Security |
| `winlogbeat` | Winlogbeat |
:::
26 changes: 21 additions & 5 deletions docs/configure/content-set/navigation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,11 @@ Optional key. Defaults to `false`. When enabled turns soft line endings in the m

### `external_hosts`

% Or remove this section altogether?
```{applies_to}
product: deprecated 0.11.0
```

All links to external hosts must be declared in this section of `docset.yml`.

Example:
Expand Down Expand Up @@ -89,6 +94,17 @@ exclude:
- '_*.md'
```

### `products`

Products that are covered in all pages in the doc set.

:::{include} /_snippets/products-list.md
:::

`products` can also be defined in the [page level frontmatter](/syntax/frontmatter.md#products).
If you define `products` in a page's Markdown file and the `docset.yml` file also includes `products`, docs-builder will combine the two lists.
You can _not_ override doc set level `products` at the page level.

### `toc`

Defines the table of contents (navigation) for the content set. A minimal toc is:
Expand All @@ -107,7 +123,7 @@ The TOC in principle follows the directory structure on disk.
- folder: subsection
```

If a folder does not explicitly define `children` all markdown files within that folder are included automatically
If a folder does not explicitly define `children` all Markdown files within that folder are included automatically

If a folder does define `children` all markdown files within that folder have to be included. `docs-builder` will error if it detects dangling documentation files.

Expand All @@ -122,8 +138,8 @@ If a folder does define `children` all markdown files within that folder have to

#### Virtual grouping

A `file` element may include children to create a virtual grouping that
does not match the directory structure.
A `file` element may include children to create a virtual grouping that
does not match the directory structure.

```yaml
...
Expand All @@ -137,7 +153,7 @@ A `file` may only select siblings and more deeply nested files as its children.

#### Hidden files

A hidden file can be declared in the TOC.
A hidden file can be declared in the TOC.
```yaml
- hidden: developer-pages.md
```
Expand Down Expand Up @@ -176,6 +192,6 @@ See [Attributes](./attributes.md) to learn more.

As a rule, each `docset.yml` file can only be included once in the assembler. This prevents us from accidentally duplicating pages in the docs. However, there are times when you want to split content sets and include them partially in different areas of the TOC. That's what `toc.yml` files are for. These files split one documentation set into multiple “sub-TOCs,” each mapped to a different navigation node.

A `toc.yml` file may only declare a nested [TOC](#toc), other options are ignored.
A `toc.yml` file may only declare a nested [TOC](#toc), other options are ignored.

A `toc.yml` may not link to further nested `toc.yml` files. Doing so will result in an error
65 changes: 25 additions & 40 deletions docs/syntax/frontmatter.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,43 +43,28 @@ See [](./applies.md)
## Products

The products frontmatter is a list of products that the page relates to.
This is used for the "Products" filter in the Search UI.

The products frontmatter is a list of objects, each object has an `id` field.

| Product ID | Product Name |
|---------------------------------------------|-----------------------------------------------|
| `apm` | APM |
| `apm-agent` | APM Agent |
| `auditbeat` | Auditbeat |
| `beats` | Beats |
| `cloud-control-ecctl` | Elastic Cloud Control ECCTL |
| `cloud-enterprise` | Elastic Cloud Enterprise |
| `cloud-hosted` | Elastic Cloud Hosted |
| `cloud-kubernetes` | Elastic Cloud Kubernetes |
| `cloud-serverless` | Elastic Cloud Serverless |
| `cloud-terraform` | Elastic Cloud Terraform |
| `ecs` | Elastic Common Schema (ECS) |
| `ecs-logging` | ECS Logging |
| `edot-cf` | EDOT Cloud Forwarder |
| `edot-sdk` | Elastic Distribution of OpenTelemetry SDK |
| `edot-collector` | Elastic Distribution of OpenTelemetry Collector |
| `elastic-agent` | Elastic Agent |
| `elastic-serverless-forwarder` | Elastic Serverless Forwarder |
| `elastic-stack` | Elastic Stack |
| `elasticsearch` | Elasticsearch |
| `elasticsearch-client` | Elasticsearch Client |
| `filebeat` | Filebeat |
| `fleet` | Fleet |
| `heartbeat` | Heartbeat |
| `integrations` | Integrations |
| `kibana` | Kibana |
| `logstash` | Logstash |
| `machine-learning` | Machine Learning |
| `metricbeat` | Metricbeat |
| `observability` | Elastic Observability |
| `packetbeat` | Packetbeat |
| `painless` | Painless |
| `search-ui` | Search UI |
| `security` | Elastic Security |
| `winlogbeat` | Winlogbeat |

:::{include} /_snippets/products-list.md
:::

`products` can also be defined in the [`docset.yml` file](/configure/content-set/navigation.md#products).
If you define `products` in a page's Markdown file and the `docset.yml` file also includes `products`, docs-builder will combine the two lists.
You can _not_ override doc set level `products` at the page level.

:::{tip}
The purpose of the `products` field is different from `applies_to`: the former provides metadata to elastic.co search engines, while the latter indicates feature availability and applicability.

% Is this true?
For example, while a page might contain content that is _applicable_ to Elastic Cloud Serverless projects,
it might not be _about_ Elastic Cloud Serverless. In that case, you would include the `serverless` key in `applies_to`,
but _not_ include the `cloud-serverless` ID in `products`:
Comment on lines +57 to +60
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we perhaps replace this with a small table of When to use vs When not to use?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense.

But the original reason was that products are only used for the search.elastic.co filters. At the time of implementation, we did not know how applies_to and products relate to.

Not sure if we even know this today. (this statement is actually a good explanation) But yes, there were ongoing discussions that they are similar. same same but different.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about this guidance @elastic/docs-tech-leads?

Could we perhaps replace this with a small table of When to use vs When not to use?

Can you be more specific about what the table would include? General guidelines? Multiple examples? Something else?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@colleenmcginnis Examples, I would say. Just a suggestion though, this is not blocking the PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a tricky example. I would add the corresponding product(s) in all cases.

Meaning that "A page with content that applies to Elastic Cloud Serverless should also be identified with the "cloud-serverless" product ID", to be searchable as such using the elastic.co search. I think this is true in most cases. If the content applies to "Stack", then you'd want product ids to identify which stack components are relevant here, etc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Meaning that "A page with content that applies to Elastic Cloud Serverless should also be identified with the "cloud-serverless" product ID", to be searchable as such using the elastic.co search.

In that case wouldn't we want that to be derived automatically from the applies_to at some point, rather than requiring the duplication? Then the products would only need to include the level of product detail (components of the stack, etc) that aren't covered in the other metadata

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add the corresponding product(s) in all cases.

This would require revisiting the products metadata all pages across all repos to duplicate what's included in applies_to since that was not the guidance we used when first adding products. I'm not sure if it's worth the effort without really knowing how we want to use this metadata going forward. @KOTungseth thoughts?

It would probably also mean that filtering by search results by Elastic Cloud Serverless might not be that helpful on its own since so many pages include serverless/stateful applies_to. For example, the PRs that sparked this PR were adding cloud-serverless to (almost) all APM agent docs pages.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case wouldn't we want that to be derived automatically from the applies_to at some point

IMO, yes, except for "Stack" where the corresponding product IDs are more granular: elasticsearch, kibana, etc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But overall this seems logical, if we mark content as applicable to a context, why wouldn't we allow users to find that content when selecting that context through facets?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would require revisiting the products metadata all pages across all repos to duplicate what's included in applies_to since that was not the guidance we used when first adding products. I'm not sure if it's worth the effort without really knowing how we want to use this metadata going forward.

I agree, and I think we shouldn't change this logic before the next phase of thinking around applies_to and versioning. For now we should "just" make sure that the way we specify products ids leads to a better search experience


```yaml
products:
- id: apm
applies_to:
stack: ga
serverless:
observability: ga
```
:::
Loading