From 3c4f5b746e3654aa6e9024e230cb5a0d92367012 Mon Sep 17 00:00:00 2001 From: Tiago Queiroz Date: Thu, 4 Sep 2025 12:20:50 -0400 Subject: [PATCH 1/6] [Filebeat] add known issue entries about Filestream clean_inactive --- docs/release-notes/known-issues.md | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/docs/release-notes/known-issues.md b/docs/release-notes/known-issues.md index 6e1939bed864..9eb3f6941822 100644 --- a/docs/release-notes/known-issues.md +++ b/docs/release-notes/known-issues.md @@ -29,4 +29,22 @@ On 04/16/2025, a known issue was discovered that can cause a crash of the Event As a workaround, and to prevent crashes, Beats will ignore any filters provided when working with forwarded events on Windows Server 2025 until the issue is resolved. % **Resolved** -% On [Month/Day/Year], this issue was resolved. \ No newline at end of file +% On [Month/Day/Year], this issue was resolved. + +::: + +:::{dropdown} Filebeat's Filestream input does not validate `clean_inactive`. + +The Filestream input does not enforce the restrictions documented for +the `clean_inactive` option, thus allowing configurations that can +lead to data re-ingestion issues. + +::: + +:::{dropdown} Setting `clean_inactive: 0` in Filebeat' Filestream input will cause data to be re-ingested on every restart. + +When `clean_inactive: 0` Filestream will clean the state of all files +on start up, effectively re-ingesting all files on restart. + +**Workaround** +Disable `clean_inactive` by setting it to `-1`. From 11fe383082127e6b0c50c62f1a18a58a8ee62557 Mon Sep 17 00:00:00 2001 From: Tiago Queiroz Date: Thu, 4 Sep 2025 17:23:36 -0400 Subject: [PATCH 2/6] [Filebeat/Filestrem] Add known issue for clean_inactive --- docs/release-notes/known-issues.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/docs/release-notes/known-issues.md b/docs/release-notes/known-issues.md index 9eb3f6941822..364303f45699 100644 --- a/docs/release-notes/known-issues.md +++ b/docs/release-notes/known-issues.md @@ -34,17 +34,24 @@ As a workaround, and to prevent crashes, Beats will ignore any filters provided ::: :::{dropdown} Filebeat's Filestream input does not validate `clean_inactive`. +**Applies to**: Filebeat < 9.2.0 The Filestream input does not enforce the restrictions documented for the `clean_inactive` option, thus allowing configurations that can lead to data re-ingestion issues. +**Fixed in**: 9.2.0 ::: -:::{dropdown} Setting `clean_inactive: 0` in Filebeat' Filestream input will cause data to be re-ingested on every restart. +:::{dropdown} Setting clean_inactive: 0 in Filebeat's Filestream input will cause data to be re-ingested on every restart. +**Applies to**: Filebeat >= 8.14.0 and < 9.2.0 When `clean_inactive: 0` Filestream will clean the state of all files on start up, effectively re-ingesting all files on restart. **Workaround** -Disable `clean_inactive` by setting it to `-1`. +- For Filestream >= 8.16.0 and < 9.2.0: disable `clean_inactive` by setting `clean_inactive: -1`. +- For Filestream >= 8.14.0 and < 8.16.0 set `clean_inactive` to a very +large value. E.g 5 years: `clean_inactive: 43800h0m0s`. + +**Fixed in**: 9.2.0 From b416ea0968f3f1eb6708d48d3c949e501087c506 Mon Sep 17 00:00:00 2001 From: Tiago Queiroz Date: Fri, 5 Sep 2025 07:34:48 -0400 Subject: [PATCH 3/6] Update docs/release-notes/known-issues.md Co-authored-by: Colleen McGinnis --- docs/release-notes/known-issues.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/release-notes/known-issues.md b/docs/release-notes/known-issues.md index 364303f45699..d1c37d0e0980 100644 --- a/docs/release-notes/known-issues.md +++ b/docs/release-notes/known-issues.md @@ -43,10 +43,10 @@ lead to data re-ingestion issues. **Fixed in**: 9.2.0 ::: -:::{dropdown} Setting clean_inactive: 0 in Filebeat's Filestream input will cause data to be re-ingested on every restart. +:::{dropdown} Setting `clean_inactive` to `0` in Filebeat's Filestream input will cause data to be re-ingested on every restart. **Applies to**: Filebeat >= 8.14.0 and < 9.2.0 -When `clean_inactive: 0` Filestream will clean the state of all files +When `clean_inactive` is set to `0`, Filestream will clean the state of all files on start up, effectively re-ingesting all files on restart. **Workaround** From e6c146c7510b2662a447edad1032f7cf8e7e07a2 Mon Sep 17 00:00:00 2001 From: Tiago Queiroz Date: Fri, 5 Sep 2025 07:35:01 -0400 Subject: [PATCH 4/6] Update docs/release-notes/known-issues.md Co-authored-by: Colleen McGinnis --- docs/release-notes/known-issues.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/release-notes/known-issues.md b/docs/release-notes/known-issues.md index d1c37d0e0980..be06d3c0c95d 100644 --- a/docs/release-notes/known-issues.md +++ b/docs/release-notes/known-issues.md @@ -52,6 +52,6 @@ on start up, effectively re-ingesting all files on restart. **Workaround** - For Filestream >= 8.16.0 and < 9.2.0: disable `clean_inactive` by setting `clean_inactive: -1`. - For Filestream >= 8.14.0 and < 8.16.0 set `clean_inactive` to a very -large value. E.g 5 years: `clean_inactive: 43800h0m0s`. +large value. For example, use `clean_inactive: 43800h0m0s`, which is 5 years. **Fixed in**: 9.2.0 From 4228a5419f00cdac87a17ff5efdee7e7cb649b96 Mon Sep 17 00:00:00 2001 From: Tiago Queiroz Date: Wed, 10 Sep 2025 11:45:24 -0400 Subject: [PATCH 5/6] Review suggestions --- docs/release-notes/known-issues.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/release-notes/known-issues.md b/docs/release-notes/known-issues.md index 1172817788b4..258e64e46951 100644 --- a/docs/release-notes/known-issues.md +++ b/docs/release-notes/known-issues.md @@ -41,7 +41,7 @@ The Filestream input does not enforce the restrictions documented for the `clean_inactive` option, thus allowing configurations that can lead to data re-ingestion issues. -**Fixed in**: 9.2.0 +**Fixed planned in**: 9.2.0 by [PR #46373](https://github.com/elastic/beats/pull/46373) ::: :::{dropdown} Setting `clean_inactive` to `0` in Filebeat's Filestream input will cause data to be re-ingested on every restart. @@ -55,5 +55,5 @@ on start up, effectively re-ingesting all files on restart. - For Filestream >= 8.14.0 and < 8.16.0 set `clean_inactive` to a very large value. For example, use `clean_inactive: 43800h0m0s`, which is 5 years. -**Fixed in**: 9.2.0 +**Fixed planned in**: 9.2.0 by [PR #46373](https://github.com/elastic/beats/pull/46373) ::: From b5d180a848fd11e4388f077a925b1dff7929bb9b Mon Sep 17 00:00:00 2001 From: Colleen McGinnis Date: Wed, 10 Sep 2025 13:47:15 -0500 Subject: [PATCH 6/6] apply suggestions from code review --- docs/release-notes/known-issues.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/release-notes/known-issues.md b/docs/release-notes/known-issues.md index c715715b9aff..da494d1eb2e5 100644 --- a/docs/release-notes/known-issues.md +++ b/docs/release-notes/known-issues.md @@ -40,7 +40,7 @@ The Filestream input does not enforce the restrictions documented for the `clean_inactive` option, thus allowing configurations that can lead to data re-ingestion issues. -**Fixed planned in**: 9.2.0 by [PR #46373](https://github.com/elastic/beats/pull/46373) +**Fix planned in**: 9.2.0 by [PR #46373](https://github.com/elastic/beats/pull/46373) ::: :::{dropdown} Setting `clean_inactive` to `0` in Filebeat's Filestream input will cause data to be re-ingested on every restart. @@ -54,7 +54,7 @@ on start up, effectively re-ingesting all files on restart. - For Filestream >= 8.14.0 and < 8.16.0 set `clean_inactive` to a very large value. For example, use `clean_inactive: 43800h0m0s`, which is 5 years. -**Fixed planned in**: 9.2.0 by [PR #46373](https://github.com/elastic/beats/pull/46373) +**Fix planned in**: 9.2.0 by [PR #46373](https://github.com/elastic/beats/pull/46373) ::: :::{dropdown} Beats panic on restart when "restart_on_cert_change" is enabled on Linux