Skip to content

Commit c3d1176

Browse files
authored
kafka: document ${partition} variable (#215)
documents questdb/kafka-questdb-connector#32 also: changed default types from string to varchar since that's what questdb now uses.
1 parent b5e8c14 commit c3d1176

File tree

1 file changed

+10
-8
lines changed

1 file changed

+10
-8
lines changed

documentation/third-party-tools/kafka.md

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -287,9 +287,9 @@ Example: Consider the following Kafka message:
287287

288288
The connector will create a table with the following columns:
289289

290-
| firstname <sub>string</sub> | lastname <sub>string</sub> | age <sub>long</sub> | address_street <sub>string</sub> | address_city <sub>string</sub> |
291-
| --------------------------- | -------------------------- | ------------------- | -------------------------------- | ------------------------------ |
292-
| John | Doe | 30 | Main Street | New York |
290+
| firstname <sub>varchar</sub> | lastname <sub>varchar</sub> | age <sub>long</sub> | address_street <sub>varchar</sub> | address_city <sub>varchar</sub> |
291+
|------------------------------|-----------------------------|---------------------|-----------------------------------|---------------------------------|
292+
| John | Doe | 30 | Main Street | New York |
293293

294294
### Client configuration string
295295

@@ -568,9 +568,10 @@ table=example_table
568568
[...]
569569
```
570570

571-
The `table` options supports simple templating. You can use the following
572-
variables in the table name: `${topic}`, `${key}`. The connector will replace
573-
these variables with the actual topic name and key value from the Kafka message.
571+
The `table` option supports simple templating. You can use the following
572+
variables in the table name: `${topic}`, `${key}`, `${partition}`. The connector will replace
573+
these variables with the actual topic name, key value, and partition number from the Kafka message.
574+
574575

575576
Example:
576577

@@ -579,14 +580,15 @@ name=questdb-sink
579580
connector.class=io.questdb.kafka.QuestDBSinkConnector
580581
client.conf.string=http::addr=localhost:9000;
581582
topics=example-topic
582-
table=from_kafka_${topic}_${key}_suffix
583+
table=from_kafka_${topic}_${key}_${partition}_suffix
583584

584585
[...]
585586
```
586587

587588
The placeholder `${key}` will be replaced with the actual key value from the
588589
Kafka message. If the key is not present in the message, the placeholder will be
589-
replaced with the string `null`.
590+
replaced with the string `null`. The placeholder `${partition}` will be replaced
591+
with the Kafka partition number from which the message originates.
590592

591593
#### Table schema
592594

0 commit comments

Comments
 (0)