Skip to content

Commit 83481f6

Browse files
committed
Review feedback, split into FixedSizeListType and VariableSizeListType
1 parent cb93b27 commit 83481f6

File tree

2 files changed

+26
-11
lines changed

2 files changed

+26
-11
lines changed

LogicalTypes.md

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -259,13 +259,21 @@ The sort order for `FLOAT16` is signed (with special handling of NANs and signed
259259
### FIXED_SIZE_LIST
260260

261261
The `FIXED_SIZE_LIST` annotation represents a fixed-size list of elements
262-
of a primitive data type. It must annotate a `binary` primitive type.
262+
of a primitive data type. It must annotate a `FIXED_LEN_BYTE_ARRAY` primitive type.
263263

264-
The `binary` data is interpreted as a sequence of elements of
265-
the same primitive data type.
264+
The `FIXED_LEN_BYTE_ARRAY` data is interpreted as a fixed size sequence of
265+
elements of the same primitive data type.
266266

267267
The sort order used for `FIXED_SIZE_LIST` is undefined.
268268

269+
### VARIABLE_SIZE_LIST
270+
271+
The `VARIABLE_SIZE_LIST` annotation represents a variable-size list of elements
272+
of a primitive data type. It must annotate a `BYTE_ARRAY` primitive type.
273+
274+
The `BYTE_ARRAY` data is interpreted as a variable size sequence of elements of
275+
the same primitive data type.
276+
269277
## Temporal Types
270278

271279
### DATE

src/main/thrift/parquet.thrift

Lines changed: 15 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -289,7 +289,13 @@ struct ListType {} // see LogicalTypes.md
289289
struct EnumType {} // allowed for BYTE_ARRAY, must be encoded with UTF-8
290290
struct DateType {} // allowed for INT32
291291
struct Float16Type {} // allowed for FIXED[2], must encoded raw FLOAT16 bytes
292-
struct FixedSizeListType {} // see LogicalTypes.md
292+
struct FixedSizeListType { // allowed for FIXED_LEN_BYTE_ARRAY[num_values * width of type],
293+
1: required Type type; // see LogicalTypes.md
294+
2: required i32 num_values;
295+
}
296+
struct VariableSizeListType { // allowed for BYTE_ARRAY, see LogicalTypes.md
297+
1: required Type type;
298+
}
293299

294300
/**
295301
* Logical type to annotate a column that is always null.
@@ -398,14 +404,15 @@ union LogicalType {
398404
8: TimestampType TIMESTAMP
399405

400406
// 9: reserved for INTERVAL
401-
10: IntType INTEGER // use ConvertedType INT_* or UINT_*
402-
11: NullType UNKNOWN // no compatible ConvertedType
403-
12: JsonType JSON // use ConvertedType JSON
404-
13: BsonType BSON // use ConvertedType BSON
405-
14: UUIDType UUID // no compatible ConvertedType
406-
15: Float16Type FLOAT16 // no compatible ConvertedType
407+
10: IntType INTEGER // use ConvertedType INT_* or UINT_*
408+
11: NullType UNKNOWN // no compatible ConvertedType
409+
12: JsonType JSON // use ConvertedType JSON
410+
13: BsonType BSON // use ConvertedType BSON
411+
14: UUIDType UUID // no compatible ConvertedType
412+
15: Float16Type FLOAT16 // no compatible ConvertedType
407413
// 16: reserved for GEOMETRY
408-
17: FixedSizeListType FIXED_SIZE_LIST // no compatible ConvertedType
414+
17: FixedSizeListType FIXED_SIZE_LIST // no compatible ConvertedType
415+
18: VariableSizeListType VARIABLE_SIZE_LIST // no compatible ConvertedType
409416
}
410417

411418
/**

0 commit comments

Comments
 (0)