Skip to content

Commit 96656a5

Browse files
steveloughranalamb
andauthored
GH-561: variant schema examples to use (VARIANT(1)) (#562)
* GH-561 variant schema examples use (VARIANT) rather than (VARIANT(1)) Fix schema examples and in logical types doc declare that the version number is required. * Update LogicalTypes.md Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org> --------- Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
1 parent 285b6fd commit 96656a5

3 files changed

Lines changed: 8 additions & 7 deletions

File tree

LogicalTypes.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -571,7 +571,8 @@ type `binary`, which is also called `BYTE_ARRAY` in the Parquet thrift definitio
571571
The `VARIANT` annotated group can be used to store either an unshredded Variant
572572
value, or a shredded Variant value.
573573

574-
* The Variant group must be annotated with the `VARIANT` logical type.
574+
* The Variant group must be annotated with the `VARIANT` logical type, with the version number
575+
included in the declaration.
575576
* Both fields `value` and `metadata` must be of type `binary` (called `BYTE_ARRAY`
576577
in the Parquet thrift definition).
577578
* The `metadata` field is required and must be a valid Variant metadata component,

VariantEncoding.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -53,15 +53,15 @@ A Variant value in Parquet is represented by a group with 2 fields, named `value
5353
This is the expected unshredded representation in Parquet:
5454

5555
```
56-
optional group variant_name (VARIANT) {
56+
optional group variant_name (VARIANT(1)) {
5757
required binary metadata;
5858
required binary value;
5959
}
6060
```
6161

6262
This is an example representation of a shredded Variant in Parquet:
6363
```
64-
optional group shredded_variant_name (VARIANT) {
64+
optional group shredded_variant_name (VARIANT(1)) {
6565
required binary metadata;
6666
optional binary value;
6767
optional int64 typed_value;

VariantShredding.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ When `typed_value` is present, readers **must** reconstruct shredded values acco
4444

4545
For example, a Variant field, `measurement` may be shredded as long values by adding `typed_value` with type `int64`:
4646
```
47-
required group measurement (VARIANT) {
47+
required group measurement (VARIANT(1)) {
4848
required binary metadata;
4949
optional binary value;
5050
optional int64 typed_value;
@@ -128,7 +128,7 @@ However, at least one of the two fields must be present.
128128

129129
For example, a `tags` Variant may be shredded as a list of strings using the following definition:
130130
```
131-
optional group tags (VARIANT) {
131+
optional group tags (VARIANT(1)) {
132132
required binary metadata;
133133
optional binary value;
134134
optional group typed_value (LIST) { # must be optional to allow a null list
@@ -174,7 +174,7 @@ As a result, reads when a field is defined in both `value` and a `typed_value` s
174174

175175
For example, a Variant `event` field may shred `event_type` (`string`) and `event_ts` (`timestamp`) columns using the following definition:
176176
```
177-
optional group event (VARIANT) {
177+
optional group event (VARIANT(1)) {
178178
required binary metadata;
179179
optional binary value; # a variant, expected to be an object
180180
optional group typed_value { # shredded fields for the variant object
@@ -229,7 +229,7 @@ The `typed_value` associated with any Variant `value` field can be any shredded
229229
For example, the `event` object above may also shred sub-fields as object (`location`) or array (`tags`).
230230

231231
```
232-
optional group event (VARIANT) {
232+
optional group event (VARIANT(1)) {
233233
required binary metadata;
234234
optional binary value;
235235
optional group typed_value {

0 commit comments

Comments
 (0)