Add variant_experimental feature to parquet crate

[alamb]

Which issue does this PR close?

• Closes [Variant] Add variant feature to parquet crate #8132
• Part of [Variant] Integration tests for reading parquet w/ Variants #8084
• Follow on to [VARIANT] Initial integration tests for variant reads #8104

Rationale for this change

TLDR is we need a way to test and work out how Variant integration with the actual parquet reader/writer will look, so let's do it in the parquet crate.

Please see the essay on #8132 for background

Follow on tasks (I will file tickets for these items if we agree on this as an integration mechanism):

• Do not panic when writing VariantArray with the ArrowWriter: [Variant] writing a VariantArray to parquet panics #8296
• Add some way to write the logical annotation to parquet metadata
• Read arrays annotated with VARIANT logical type as VariantArrays in ArrowReader
• Update the variant_integration test to use VariantArray : [Variant] Integration tests for reading parquet w/ Variants #8084
• Rename variant_experimental flag to variant and remove warnings about being experimental: [Variant] Rename variant_experimental flag to variant and remove warnings about being experimental #8297

Follow up tasks that came out of this PR but do not depend on it

• [Variant] Support StringView and LargeString in ´batch_json_string_to_variant` #8145
• [Variant] Rename batch_json_string_to_variant and batch_variant_to_json_string json_to_variant #8144

What changes are included in this PR?

1. Add the variant_experimental feature to the parquet crate
2. Publicly export the variant crates
3. Add docs and examples

Are these changes tested?

Yes by new CI

Are there any user-facing changes?

This adds a new feature flag, and new

[alamb]

Being able to write this example is the largest rationale in my mind for the feature flag in the parquet crate

[alamb]

I will file a follow on ticket for this problem

[alamb]

This is kind of neat to see the data written. I looked at the resulting file using datafusion-cli and it looks like like this (note it does not have the parquet logical annotation on it)

> select * from 'parquet/variant-json.parquet';
+-----------------------------------------------------------------------------------------------------------------------------------+
| data                                                                                                                              |
+-----------------------------------------------------------------------------------------------------------------------------------+
| {metadata: 11020003076167656e616d65, value: 020200010002080c1e15416c696365}                                                       |
| {metadata: 010400040b0e1263697479616464726573736167656e616d65, value: 0203010203000e10140201000009214e657720596f726b0c190d426f62} |
| {metadata: 010000, value: 020000}                                                                                                 |
| {metadata: 010000, value: 020000}                                                                                                 |
+-----------------------------------------------------------------------------------------------------------------------------------+
4 row(s) fetched.
Elapsed 0.003 seconds.

[alamb]

I would like to know if there are any objections to this approach from other maintainers such as @Dandandan @etseidl @waynexia @tustvold and @mbrobbel

Also, other Variant contributors like @carpecodeum @scovich @mprammer @klion26 @friendlymatthew please provide any feedback if you have it

[etseidl]

No objections from me.

[friendlymatthew]

Very exciting. Thank you!

[klion26]

Thank you for pushing this forward!

[alamb]

I am hoping we can merge this after I made the RC for the next release (ideally in the next day or two)

• Release arrow-rs / parquet Minor version 56.1.0 (August 2025) #7837

[mbrobbel]

Thanks @alamb