Skip to content

Conversation

nikbyte
Copy link
Member

@nikbyte nikbyte commented May 21, 2025

Summary

This PR adds per-file headers to the event_flatstore module.
With the new header parameter, you can write an arbitrary string (e.g. CSV column names) as the very first line of every newly created or rotated log file — making it easier to work with parsers and analysis tools.

Details

  • Current behavior
    When a flatstore log file is rotated, the new file starts immediately with data lines — no header is added.

  • Problem
    Many downstream tools expect a header row (e.g. in CSV files). Manually injecting headers after rotation is error-prone and inconvenient. Letting the module write the header automatically ensures consistency.

Solution

  • New parameter
Parameter Type Default Notes
header string (empty string) Written as the first line of every new file. No trailing newline required. UTF-8 supported.

When the module opens a new file (initial, rotated, or reopened), it will prepend this string as a line if header is set.

Compatibility

  • Fully backwards-compatible — if header is not configured, behavior remains unchanged.

@razvancrainea
Copy link
Member

@nikbyte I don't like the idea of hardcoding one single header for any type of events - this will only make sense if you trigger just one single event through event_flatstore, but if you have two, that would be completely broken.
Perhaps there is a way of fetching the names of the fields that are triggered, at least for events with named fields, and dump them in the header. This information is available in the flat_raise function in the params argument.

@nikbyte
Copy link
Member Author

nikbyte commented Jun 6, 2025

@razvancrainea Thank you for your comment, I'll review it and try to find a better way.

@razvancrainea
Copy link
Member

Hey @nikbyte ! Did you have time to look into a new approach for this? Thanks!

@nikbyte nikbyte marked this pull request as draft July 15, 2025 08:40
@nikbyte
Copy link
Member Author

nikbyte commented Jul 15, 2025

Hi @razvancrainea . I haven't had time to look at it yet, but I definitely have plans to redo it better way in the nearest future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants