You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/en/blog/features/variant.md
+12-19Lines changed: 12 additions & 19 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -112,11 +112,7 @@ The query engine decides which fields to shred based on access patterns and work
112
112
113
113
### Examples of Shredded Parquet Schemas
114
114
115
-
If a field's value matches the shredded type, it is stored in the typed column `typed_value`. If a field's value has a different type, it remains in the `value` binary column using standard Variant encoding.
116
-
117
-
#### Example 1: Shredding to String Type
118
-
119
-
The Variant values are shredded to string type.
115
+
The following example shows shredding non nested Variants. In this case, the writer chose to shred String values as the `typed_value` column. Rows which do not contain strings are stored in the `value` column, with the binary variant encoding.
#### Example 2: Shredding to Object with Typed Fields
140
-
141
-
The Variant values are shredded to an object with `user_id` field of integer type and `type` field of string type.
142
-
135
+
Shredding nested variants is similar, with the shredding applied recursively, as shown in the following example. In this case, the `userId` field is shredded as an integer, and stored as two columns: in `typed_value.userId.typed_value` when the value is integer and as a variant in `typed_value.userId.value` otherwise. Similarly, the `eType` field is shredded as a string and stored in `typed_value.eType.typed_value` and `typed_value.eType.value`.
143
136
```parquet
144
137
optional group EVENT_DATA (VARIANT(1)) = 1 {
145
138
required binary metadata; # variant metadata
146
139
optional binary value; # non-shredded value
147
140
optional group typed_value {
148
-
required group user_id { # user_id field
141
+
required group userId { # userId field
149
142
optional binary value; # non-shredded value
150
143
optional int32 typed_value; # the shredded value
151
144
}
152
-
required group type { # type field
145
+
required group eType { # eType field
153
146
optional binary value; # non-shredded value
154
147
optional binary typed_value (STRING); # the shredded value
155
148
}
156
149
}
157
150
}
158
151
```
159
152
160
-
**Encoding Table:**
153
+
**The table below illustrates how the data is stored:**
0 commit comments