SIGN IN SIGN UP
apache / spark UNCLAIMED

Apache Spark - A unified analytics engine for large-scale data processing

0 0 1103 Scala

[SPARK-47890][CONNECT][PYTHON] Add variant functions to Scala and Python

### What changes were proposed in this pull request?

Add the following functions:

- is_variant_null
- variant_get
- try_variant_get
- schema_of_variant
- schema_of_variant_agg

to:

- Scala API
- Python API
- Spark Connect Scala Client
- Spark Connect Python Client

### Why are the changes needed?

Improve the usability of variant functions.

### Does this PR introduce _any_ user-facing change?

Yes, new functions

### How was this patch tested?

Unit tests.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #46123 from chenhao-db/variant_expression_alias.

Authored-by: Chenhao Li <chenhao.li@databricks.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
C
Chenhao Li committed
79a1fa4b84dd79f8c6aeab70047f54fae1138b30
Parent: e1432ef
Committed by Hyukjin Kwon <gurwls223@apache.org> on 4/22/2024, 7:31:22 AM