Skip to content

Commit 1be6bcb

Browse files
Merge branch 'fluent:master' into feature/daemon-out-pgsql
2 parents 460af6d + 969685b commit 1be6bcb

File tree

18 files changed

+404
-116
lines changed

18 files changed

+404
-116
lines changed

SUMMARY.md

Lines changed: 38 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -25,16 +25,16 @@
2525
* [Build with static configuration](installation/downloads/source/build-with-static-configuration.md)
2626
* [Linux packages](installation/downloads/linux.md)
2727
* [Amazon Linux](installation/downloads/linux/amazon-linux.md)
28-
* [Rocky Linux and Alma Linux ](installation/downloads/linux/alma-rocky.md)
29-
* [Red Hat and CentOS](installation/downloads/linux/redhat-centos.md)
28+
* [Buildroot embedded Linux](installation/downloads/linux/buildroot-embedded-linux.md)
3029
* [Debian](installation/downloads/linux/debian.md)
31-
* [Ubuntu](installation/downloads/linux/ubuntu.md)
3230
* [Raspbian and Raspberry Pi](installation/downloads/linux/raspbian-raspberry-pi.md)
31+
* [Red Hat and CentOS](installation/downloads/linux/redhat-centos.md)
32+
* [Rocky Linux and Alma Linux ](installation/downloads/linux/alma-rocky.md)
33+
* [Ubuntu](installation/downloads/linux/ubuntu.md)
3334
* [Yocto embedded Linux](installation/downloads/linux/yocto-embedded-linux.md)
34-
* [Buildroot embedded Linux](installation/downloads/linux/buildroot-embedded-linux.md)
35-
* [Docker](installation/downloads/docker.md)
36-
* [Containers on AWS](installation/downloads/aws-container.md)
3735
* [Amazon EC2](installation/downloads/amazon-ec2.md)
36+
* [Containers on AWS](installation/downloads/aws-container.md)
37+
* [Docker](installation/downloads/docker.md)
3838
* [Kubernetes](installation/downloads/kubernetes.md)
3939
* [macOS](installation/downloads/macos.md)
4040
* [Windows](installation/downloads/windows.md)
@@ -44,36 +44,37 @@
4444

4545
* [Configure Fluent Bit](administration/configuring-fluent-bit.md)
4646
* [YAML configuration](administration/configuring-fluent-bit/yaml.md)
47+
* [Configuration file](administration/configuring-fluent-bit/yaml/configuration-file.md)
48+
* [Environment variables](administration/configuring-fluent-bit/yaml/environment-variables-section.md)
49+
* [Includes](administration/configuring-fluent-bit/yaml/includes-section.md)
4750
* [Service](administration/configuring-fluent-bit/yaml/service-section.md)
4851
* [Parsers](administration/configuring-fluent-bit/yaml/parsers-section.md)
4952
* [Multiline parsers](administration/configuring-fluent-bit/yaml/multiline-parsers-section.md)
5053
* [Pipeline](administration/configuring-fluent-bit/yaml/pipeline-section.md)
5154
* [Plugins](administration/configuring-fluent-bit/yaml/plugins-section.md)
5255
* [Upstream servers](administration/configuring-fluent-bit/yaml/upstream-servers-section.md)
53-
* [Environment variables](administration/configuring-fluent-bit/yaml/environment-variables-section.md)
54-
* [Includes](administration/configuring-fluent-bit/yaml/includes-section.md)
5556
* [Classic mode](administration/configuring-fluent-bit/classic-mode.md)
56-
* [Format and schema](administration/configuring-fluent-bit/classic-mode/format-schema.md)
57-
* [Configuration file](administration/configuring-fluent-bit/classic-mode/configuration-file.md)
58-
* [Variables](administration/configuring-fluent-bit/classic-mode/variables.md)
57+
* [Configuration file](administration/configuring-fluent-bit/classic-mode/configuration-file.md)
5958
* [Commands](administration/configuring-fluent-bit/classic-mode/commands.md)
59+
* [Format and schema](administration/configuring-fluent-bit/classic-mode/format-schema.md)
60+
* [Record accessor syntax](administration/configuring-fluent-bit/classic-mode/record-accessor.md)
6061
* [Upstream servers](administration/configuring-fluent-bit/classic-mode/upstream-servers.md)
61-
* [Record accessor syntax](administration/configuring-fluent-bit/classic-mode/record-accessor.md)
62-
* [Unit Sizes](administration/configuring-fluent-bit/unit-sizes.md)
62+
* [Variables](administration/configuring-fluent-bit/classic-mode/variables.md)
6363
* [Multiline parsing](administration/configuring-fluent-bit/multiline-parsing.md)
64-
* [TLS](administration/transport-security.md)
65-
* [Buffering and storage](administration/buffering-and-storage.md)
64+
* [Unit Sizes](administration/configuring-fluent-bit/unit-sizes.md)
65+
* [AWS credentials](administration/aws-credentials.md)
6666
* [Backpressure](administration/backpressure.md)
67-
* [Scheduling and retries](administration/scheduling-and-retries.md)
68-
* [Networking](administration/networking.md)
67+
* [Buffering and storage](administration/buffering-and-storage.md)
68+
* [Hot reload](administration/hot-reload.md)
69+
* [HTTP proxy](administration/http-proxy.md)
6970
* [Memory management](administration/memory-management.md)
7071
* [Monitoring](administration/monitoring.md)
7172
* [Multithreading](administration/multithreading.md)
72-
* [HTTP proxy](administration/http-proxy.md)
73-
* [Hot reload](administration/hot-reload.md)
74-
* [Troubleshooting](administration/troubleshooting.md)
73+
* [Networking](administration/networking.md)
7574
* [Performance tips](administration/performance.md)
76-
* [AWS credentials](administration/aws-credentials.md)
75+
* [Scheduling and retries](administration/scheduling-and-retries.md)
76+
* [TLS](administration/transport-security.md)
77+
* [Troubleshooting](administration/troubleshooting.md)
7778

7879
## Local testing
7980

@@ -91,10 +92,11 @@
9192
* [Dummy](pipeline/inputs/dummy.md)
9293
* [eBPF](pipeline/inputs/ebpf.md)
9394
* [Elasticsearch](pipeline/inputs/elasticsearch.md)
94-
* [Exec WASI](pipeline/inputs/exec-wasi.md)
9595
* [Exec](pipeline/inputs/exec.md)
96+
* [Exec WASI](pipeline/inputs/exec-wasi.md)
9697
* [Fluent Bit metrics](pipeline/inputs/fluentbit-metrics.md)
9798
* [Forward](pipeline/inputs/forward.md)
99+
* [GPU metrics](pipeline/inputs/gpu-metrics.md)
98100
* [Head](pipeline/inputs/head.md)
99101
* [Health](pipeline/inputs/health.md)
100102
* [HTTP](pipeline/inputs/http.md)
@@ -124,25 +126,25 @@
124126
* [TCP](pipeline/inputs/tcp.md)
125127
* [Thermal](pipeline/inputs/thermal.md)
126128
* [UDP](pipeline/inputs/udp.md)
127-
* [Windows Event logs (winevtlog)](pipeline/inputs/windows-event-log-winevtlog.md)
128129
* [Windows Event logs (winlog)](pipeline/inputs/windows-event-log.md)
130+
* [Windows Event logs (winevtlog)](pipeline/inputs/windows-event-log-winevtlog.md)
129131
* [Windows exporter metrics](pipeline/inputs/windows-exporter-metrics.md)
130132
* [Parsers](pipeline/parsers.md)
131133
* [Configuring parsers](pipeline/parsers/configuring-parser.md)
134+
* [Decoders](pipeline/parsers/decoders.md)
132135
* [JSON](pipeline/parsers/json.md)
133-
* [Regular expression](pipeline/parsers/regular-expression.md)
134-
* [LTSV](pipeline/parsers/ltsv.md)
135136
* [Logfmt](pipeline/parsers/logfmt.md)
136-
* [Decoders](pipeline/parsers/decoders.md)
137+
* [LTSV](pipeline/parsers/ltsv.md)
138+
* [Regular expression](pipeline/parsers/regular-expression.md)
137139
* [Processors](pipeline/processors.md)
140+
* [Conditional processing](pipeline/processors/conditional-processing.md)
138141
* [Content modifier](pipeline/processors/content-modifier.md)
142+
* [Filters as processors](pipeline/processors/filters.md)
139143
* [Labels](pipeline/processors/labels.md)
140144
* [Metrics selector](pipeline/processors/metrics-selector.md)
141145
* [OpenTelemetry envelope](pipeline/processors/opentelemetry-envelope.md)
142146
* [Sampling](pipeline/processors/sampling.md)
143147
* [SQL](pipeline/processors/sql.md)
144-
* [Filters as processors](pipeline/processors/filters.md)
145-
* [Conditional processing](pipeline/processors/conditional-processing.md)
146148
* [Filters](pipeline/filters.md)
147149
* [AWS metadata](pipeline/filters/aws-metadata.md)
148150
* [CheckList](pipeline/filters/checklist.md)
@@ -169,20 +171,21 @@
169171
* [Buffering](pipeline/buffering.md)
170172
* [Routing](pipeline/router.md)
171173
* [Outputs](pipeline/outputs.md)
174+
* [Azure Log Analytics](pipeline/outputs/azure.md)
175+
* [Azure Blob](pipeline/outputs/azure_blob.md)
176+
* [Azure Data Explorer](pipeline/outputs/azure_kusto.md)
177+
* [Azure Logs Ingestion API](pipeline/outputs/azure_logs_ingestion.md)
172178
* [Amazon CloudWatch](pipeline/outputs/cloudwatch.md)
173179
* [Amazon Kinesis Data Firehose](pipeline/outputs/firehose.md)
174180
* [Amazon Kinesis Data Streams](pipeline/outputs/kinesis.md)
175181
* [Amazon S3](pipeline/outputs/s3.md)
176182
* [Apache SkyWalking](pipeline/outputs/skywalking.md)
177-
* [Azure Blob](pipeline/outputs/azure_blob.md)
178-
* [Azure Data Explorer](pipeline/outputs/azure_kusto.md)
179-
* [Azure Log Analytics](pipeline/outputs/azure.md)
180-
* [Azure Logs Ingestion API](pipeline/outputs/azure_logs_ingestion.md)
181183
* [Counter](pipeline/outputs/counter.md)
182184
* [Dash0](pipeline/outputs/dash0.md)
183185
* [Datadog](pipeline/outputs/datadog.md)
184186
* [Dynatrace](pipeline/outputs/dynatrace.md)
185187
* [Elasticsearch](pipeline/outputs/elasticsearch.md)
188+
* [Exit](pipeline/outputs/exit.md)
186189
* [File](pipeline/outputs/file.md)
187190
* [Flow counter](pipeline/outputs/flowcounter.md)
188191
* [Forward](pipeline/outputs/forward.md)
@@ -199,10 +202,11 @@
199202
* [New Relic](pipeline/outputs/new-relic.md)
200203
* [Null](pipeline/outputs/null.md)
201204
* [Observe](pipeline/outputs/observe.md)
205+
* [Oracle Cloud Infrastructure Logging Analytics](pipeline/outputs/oci-logging-analytics.md)
202206
* [OpenObserve](pipeline/outputs/openobserve.md)
203207
* [OpenSearch](pipeline/outputs/opensearch.md)
204208
* [OpenTelemetry](pipeline/outputs/opentelemetry.md)
205-
* [Oracle Cloud Infrastructure Logging Analytics](pipeline/outputs/oci-logging-analytics.md)
209+
* [Output Formats](pipeline/outputs/output_formats.md)
206210
* [Parseable](pipeline/outputs/parseable.md)
207211
* [PostgreSQL](pipeline/outputs/postgresql.md)
208212
* [Prometheus exporter](pipeline/outputs/prometheus-exporter.md)
@@ -233,6 +237,7 @@
233237
* [C library API](development/library_api.md)
234238
* [Ingest records manually](development/ingest-records-manually.md)
235239
* [Golang output plugins](development/golang-output-plugins.md)
240+
* [MessagePack format](development/msgpack-format.md)
236241
* [Wasm filter plugins](development/wasm-filter-plugins.md)
237242
* [Wasm input plugins](development/wasm-input-plugins.md)
238243
* [Contribution guide for beginners](development/developer-guide.md)

administration/configuring-fluent-bit.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
Fluent Bit supports two configuration formats:
44

55
- [YAML](./configuring-fluent-bit/yaml.md): Standard configuration format as of v3.2.
6-
- [Classic mode](./configuring-fluent-bit/classic-mode/README.md): To be deprecated at the end of 2026.
6+
- [Classic mode](./configuring-fluent-bit/classic-mode.md): To be deprecated at the end of 2026.
77

88
## Command line interface
99

administration/configuring-fluent-bit/classic-mode/README.md

Lines changed: 0 additions & 1 deletion
This file was deleted.

administration/configuring-fluent-bit/classic-mode/configuration-file.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ The `Service` section defines global properties of the service. The following ke
3636
| `scheduler.base` | Set a base of exponential backoff. Supported in v1.8.7 and greater. | `5` |
3737
| `json.convert_nan_to_null` | If enabled, `NaN` converts to `null` when Fluent Bit converts `msgpack` to `json`. | `false` |
3838
| `sp.convert_from_str_to_num` | If enabled, Stream processor converts from number string to number type. | `true` |
39+
| `windows.maxstdio` | If specified, the limit of stdio is adjusted. Only provided for Windows. From 512 to 2048 is allowed. | `512` |
3940

4041
The following is an example of a `SERVICE` section:
4142

@@ -156,4 +157,4 @@ Wildcard character (`*`) supports including multiple files. For example:
156157
@INCLUDE input_*.conf
157158
```
158159

159-
Files matching the wildcard character are included unsorted. If plugin ordering between files needs to be preserved, the files should be included explicitly.
160+
Files matching the wildcard character are included unsorted. If plugin ordering between files needs to be preserved, the files should be included explicitly.

administration/configuring-fluent-bit/yaml/configuration-file.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,8 @@ The `service` section defines the global properties of the service. The Service
8181
| `scheduler.cap` | Set a maximum retry time in seconds. Supported from v1.8.7. | `2000` |
8282
| `scheduler.base` | Sets the base of exponential backoff. Supported from v1.8.7. | `5` |
8383
| `json.convert_nan_to_null` | If enabled, NaN is converted to null when Fluent Bit converts `msgpack` to JSON. | `false` |
84-
| `sp.convert_from_str_to_num` | If enabled, Stream processor converts from number string to number type. | `true |
84+
| `sp.convert_from_str_to_num` | If enabled, Stream processor converts from number string to number type. | `true` |
85+
| `windows.maxstdio` | If specified, the limit of stdio is adjusted. Only provided for Windows. From 512 to 2048 is allowed. | `512` |
8586

8687
The following is an example of a `service` section:
8788

pipeline/filters/rewrite-tag.md

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -176,6 +176,54 @@ $ fluent-bit -c example.conf
176176
...
177177
[0] from.test_tag.new.fluent.bit.out: [1580436933.000050569, {"tool"=>"fluent", "sub"=>{"s1"=>{"s2"=>"bit"}}}]
178178
```
179+
## Configuration example with multiple rules
180+
In cases using multiple rules, the rules are passed through in order until one matches. With `AND_COMBINE` using the value `true` as optional fifth component, the rule is combined with the following rule like an 'and' combination. If the first and following rule match, the message is retagged with the tag in the last matched rule.
181+
182+
An `AND_COMBINE` in the last rule is ignored.
183+
```text
184+
[SERVICE]
185+
Flush 5
186+
Log_Level info
187+
188+
[INPUT]
189+
Name tail
190+
Tag tail
191+
Path /var/tmp/loginput.txt
192+
193+
[FILTER]
194+
Name rewrite_tag
195+
Match tail
196+
Rule $log ^(1)$ newtag_or false
197+
Rule $log ^(.*and)$ newtag_and_1 false true
198+
Rule $log ^(1.*)$ newtag_and_2 false
199+
Rule $log ^(42)$ newtag_or false
200+
Rule $log ^(9)$ newtag_and_3 false true
201+
202+
[OUTPUT]
203+
Name stdout
204+
Match *
205+
```
206+
Use the input file `/var/tmp/loginput.txt`:
207+
```text
208+
1
209+
2
210+
3
211+
9
212+
10and
213+
10
214+
42
215+
```
216+
217+
The log messages will be rewritten:
218+
```
219+
fluent-bit_1 | [0] tail: [1596050753.241336500, {"log"=>"2"}]
220+
fluent-bit_1 | [1] tail: [1596050753.241356700, {"log"=>"3"}]
221+
fluent-bit_1 | [2] tail: [1596050753.241410100, {"log"=>"10"}]
222+
fluent-bit_1 | [0] newtag_or: [1596050753.237370100, {"log"=>"1"}]
223+
fluent-bit_1 | [1] newtag_or: [1596050753.241427200, {"log"=>"42"}]
224+
fluent-bit_1 | [0] newtag_and_3: [1596050753.241374500, {"log"=>"9"}]
225+
fluent-bit_1 | [0] newtag_and_2: [1596050753.241392800, {"log"=>"10and"}]
226+
```
179227

180228
## Monitoring
181229

0 commit comments

Comments
 (0)