in_calyptia_fleet: add support for net.* properties for the upstream connection. #10998
Conversation
Signed-off-by: Phillip Whelan <[email protected]>
Signed-off-by: Phillip Whelan <[email protected]>
Signed-off-by: Phillip Whelan <[email protected]>
WalkthroughPropagates input network properties into Calyptia custom plugin (fleet input and cloud output) and Calyptia Fleet input, adds a configurable Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant User as User Config
participant CP as custom_calyptia
participant Fleet as Fleet Input
participant Cloud as Cloud Output
rect rgba(220,240,255,0.4)
note over CP: Plugin init
User->>CP: Load config (machine_id, net_properties)
CP->>CP: register plugin (.flags = FLB_CUSTOM_NET_CLIENT)
CP->>Fleet: set_fleet_input_properties(...)
loop for each net_property
CP->>Fleet: apply property (key=value)
end
CP->>Cloud: setup_cloud_output(...)
loop for each net_property
CP->>Cloud: set property (key=value)
end
end
sequenceDiagram
autonumber
participant IF as in_calyptia_fleet
participant UP as Upstream (HTTP)
participant Cfg as Fleet Config
rect rgba(235,255,235,0.4)
note over IF: Initialization
IF->>UP: initialize upstream
IF->>UP: flb_input_upstream_set(ctx->u, ctx->ins)
IF->>Cfg: fleet_config_get_properties(base_props)
IF->>Cfg: fleet_config_get_properties(c_ins->net_properties)
Cfg-->>IF: merged properties
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Suggested reviewers
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
plugins/custom_calyptia/calyptia.c (2)
218-220: Remove unused variabletmp.The variable
tmpis declared but never used in theset_fleet_input_propertiesfunction. Onlyheadandkeyvalare needed for the net_properties iteration at lines 262-266.Apply this diff to remove the unused variable:
struct mk_list *head; - struct mk_list *tmp; struct flb_kv *keyval;
280-281: Remove unused variabletmp.The variable
tmpis declared but never used in thesetup_cloud_outputfunction. Onlykeyvalis needed for the net_properties iteration at lines 369-373.Apply this diff to remove the unused variable:
- struct mk_list *tmp; struct flb_kv *keyval;
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
plugins/custom_calyptia/calyptia.c(6 hunks)plugins/in_calyptia_fleet/in_calyptia_fleet.c(2 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
plugins/in_calyptia_fleet/in_calyptia_fleet.c (1)
src/flb_input.c (1)
flb_input_upstream_set(2207-2226)
plugins/custom_calyptia/calyptia.c (2)
src/flb_input.c (1)
flb_input_set_property(557-774)src/flb_output.c (1)
flb_output_set_property(834-1056)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (30)
- GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit, x64, x64-windows-static, 3.31.6)
- GitHub Check: pr-windows-build / call-build-windows-package (Windows 32bit, x86, x86-windows-static, 3.31.6)
- GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit (Arm64), amd64_arm64, -DCMAKE_SYSTEM_NAME=Windows -DCMA...
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_ARROW=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_COVERAGE=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, clang, clang++)
- GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, gcc, g++)
- GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, clang, clang++)
- GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-22.04, clang-12)
- GitHub Check: pr-compile-without-cxx (3.31.6)
- GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-22.04, clang-12)
- GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-24.04, clang-14)
- GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-24.04, clang-14)
- GitHub Check: pr-compile-centos-7
- GitHub Check: PR - fuzzing test
🔇 Additional comments (6)
plugins/in_calyptia_fleet/in_calyptia_fleet.c (2)
1886-1886: LGTM! Net properties correctly emitted to fleet config.The addition properly includes network properties (net.*) in the generated fleet configuration alongside regular properties. This enables the network settings to be persisted and applied to fleet components.
2547-2549: LGTM! Upstream network settings properly propagated.The call to
flb_input_upstream_setcorrectly propagates the input instance's network settings (parsed from net.* properties) to the upstream connection. This is the standard pattern for applying network configuration to upstream connections.plugins/custom_calyptia/calyptia.c (4)
262-266: LGTM! Network properties correctly propagated to fleet input.The iteration properly applies each network property from the custom plugin instance to the fleet input. This enables users to configure network settings like
net.dns.modeornet.connect_timeouton the custom Calyptia plugin, which then flow through to the underlying fleet input.
369-373: LGTM! Network properties correctly propagated to cloud output.The iteration properly applies each network property from the custom plugin instance to the Calyptia cloud output. This ensures network configuration settings are consistently applied across both the fleet input and cloud output components.
749-753: LGTM! Machine ID configuration option added.The new
machine_idconfiguration option allows users to provide a custom machine identifier instead of using the auto-generated one. This is useful for fleet deployments where stable, predetermined machine IDs are required.
812-812: LGTM! Plugin properly registered as network client.The
FLB_CUSTOM_NET_CLIENTflag is necessary for the configuration parser to recognize and store net.* properties in the plugin instance'snet_propertieslist. This enables the net.* property support added throughout this PR.
|
Here is the debug.log: debug.log Here is the valgrind.log: This log exhibits the same |
… custom calyptia plugin. Signed-off-by: Phillip Whelan <[email protected]>
|
There was a flake in the I restarted the test for now but it would be a good idea to investigate later. At first glance it does not seem to be due to noisy neighbors. |
1 participant
Summary
Add support for net.* properties for
in_calyptia_fleet, ie:net.dns.modeto TCP for some networks or setting thenet.dns.resolverto use the legacy DNS resolver.Description
This PR hooks up the support for net* properties for the upstream connection the
in_calyptia_fleetplugin uses. Among the main reasons for this is for the support for setting the dns resolver.This requires a change to the
custom_calyptiaplugin as well. With this PR it is now possible to configure the network settings like so:[CUSTOM] Name calyptia Fleet_Name test-fleet net.dns.resolver legacy net.connect_timeout 60 API_KEY ********Enter
[N/A]in the box, if an item is not applicable to your change.Testing
Before we can approve your change; please submit the following in a comment:
If this is a change to packaging of containers or native binaries then please confirm it works for all targets.
Documentation
Backporting
Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.
Summary by CodeRabbit