ShreddingAdvice
One path worth promoting to a typed column in a Parquet Variant shredding schema, with the evidence behind the recommendation.
Properties
distinct values, exact below the sketch's sparse limit and an estimate above it. See InferredField.distinctIsExact.
how much of the present values are JSON null.
the Parquet physical type the observations point at — INT64, DOUBLE, DECIMAL(p, s), BINARY (UTF8), BOOLEAN. Named in Parquet's vocabulary rather than the engine's, because the whole point of this object is to be read by somebody writing a schema.
the path, with array indices collapsed. See CatalogPath.
true when the path holds values outside parquetType's family, so the shredded column needs its untyped variant_value fallback populated rather than being a pure typed column. This is the field a hand-written schema gets wrong.
how much of the path's observed values are of parquetType's family. Below 1.0 the residual values still have to go somewhere — see residual.