Timezone for iceberg timestamptz #1103
Conversation
src/Core/Settings.cpp
Outdated
| DECLARE(Bool, export_merge_tree_part_overwrite_file_if_exists, false, R"( | ||
| Overwrite file if it already exists when exporting a merge tree part | ||
| )", 0) \ | ||
| DECLARE(Timezone, timezone_for_iceberg_timestamptz, "UTC", R"( |
There was a problem hiding this comment.
Would it make sense to prepend all iceberg related settings with iceberg_? So this one would become iceberg_timezone_for_timestamptz
I understand that there is no uniformity and half of the settings begin with iceberg_, and rest are in complete disarray, but let's try to keep things in order
| using Node = ActionsDAG::Node; | ||
|
|
||
| public: | ||
| IcebergSchemaProcessor(ContextPtr context_) : WithContext(context_) {} |
| assert "Invalid time zone: Foo/Bar" in node.query_and_get_error(f""" | ||
| SELECT * FROM {CATALOG_NAME}.`{root_namespace}.{table_name}` | ||
| SETTINGS timezone_for_iceberg_timestamptz='Foo/Bar' | ||
| """) |
There was a problem hiding this comment.
I suggest checking timezone of the column with https://clickhouse.com/docs/sql-reference/functions/date-time-functions#timezoneOf , something like this:
for timezone in ('UTC', 'Europe/Berlin', 'Asia/Istanbul'):
assert node.query(f"""
SELECT timezoneOf(timestamp), timezoneOf(timestamptz),
SETTINGS timezone_for_iceberg_timestamptz='{timezone}'
""") == f"{timezone}\t{timezone}"
There was a problem hiding this comment.
I check different cases here. Added comments.
There was a problem hiding this comment.
yeah, but these tests check for serialization of the value to the string, NOT the timezone of the value. The setting may affect date time values in multiple ways:
- changing the numerical value, but leaving timezone to UTC
- leaving the numerical value intact, but changing the timezone value (=proper way to do it)
And these tests make no distinction between two... Also having proper timezone value is critical for determining day boundaries. Same UTC time value could be different days at different locations.
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Setting
timezone_for_iceberg_timestamptzfor Iceberg timestamptz type.Documentation entry for user-facing changes
Solved ClickHouse#86641
Setting
timezone_for_iceberg_timestamptzfor Iceberg timestamptz type.Possible values:
Europe/Berlin,UTCor `ZuluDefault value is
UTC.CI/CD Options
Exclude tests:
Regression jobs to run: