[EPIC][Parquet] Finalize Variant Type support in Parquet

[alamb]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

Parquet adopted the Variant type from Spark VariantEncoding.md and we completed an initial implementation of Variant:

• [EPIC] [Parquet] Implement Variant type support in Parquet #6736

This support includes:

1. APIs for reading/writing Variant encoded data (parquet-variant crate)
2. APIs for converting to/from JSON (parquet-variant-json crate)
3. APIs for working with Arrow arrays of Variants, including kernels cast_to_variant, variant_get and shred_variant

Describe the solution you'd like
There are still several areas for additional work, with tickets gathered below

API Finalization

• [Variant] Rename variant_experimental flag to variant and remove warnings about being experimental #8297
• [Variant] Allow configurable JSON format for specialized types #8605
• [Variant] Remove ceremony of going from list of Variant to VariantArray #8606
• [Variant] Remove potential panics when probing VariantArray #8609
• [Variant] impl PartialEq for VariantArray #8610
• [Variant] Make VariantArray iterable #8612

Shredding Support

• [Variant] Support Shredded Objects in variant_get: untyped path access #8151
• [Variant] extend shredded null handling for arrays #8400
• [Variant] Support Shredded Objects in variant_get: access as Some(DataType::Struct) (nested shredding) #8153
• [Variant] Support Uuid/FixedSizeBinary(16) shredding #8665
• [Variant] [Shredding] Support typed_access for Decimal128 #8332
• [Variant] [Shredding] Support typed_access for Time64(Microsecond) #8334
• [Variant] [Shredding] Support typed_access for Struct #8336
• [Variant] [Shredding] Support typed_access for List #8337

Full variant_get Support

• [Variant] Support Shredded Objects in variant_get: Access as VARIANT (Unshredding) #8154
• [Variant] Support Shredded Lists/Array in variant_get #8082
• [Variant] Casting errors behavior support in variant_get #8086
• [Variant] Support typed access for other types in variant_get #8087

** Full variant_to_arrow support

• [Variant] variant_to_arrow types support #8477

Infrastructure

• [Variant] Support BinaryArray/LargeBinaryArray in addition to BinaryViewArray for Variant #8387
• [Variant] Overly aggressive inference of UUID values #8420
• [Variant] Optimize the object header generation logic in ObjectBuilder::finish #7978
• [Variant] Revisit validation cost of infallible iterators #7711
• [Variant] VariantBuilder (wrongly?) accepts 0+ variant values #7870
• [Variant] Avoiding extra splice in ObjectBuilder::finish if possible #7960