SIGN IN SIGN UP
apache / spark UNCLAIMED

Apache Spark - A unified analytics engine for large-scale data processing

0 0 313 Scala

[SPARK-50685][PYTHON] Improve Py4J performance by leveraging getattr

### What changes were proposed in this pull request?

This Pr proposes to improve Py4J performance by leveraging `getattr`, see also https://github.com/apache/spark/pull/46809

This PR fixes Core, SQL, ML and Structured Streaming. Tests codes, MLLib and DStream are not affected.

### Why are the changes needed?

To reduce the overhead of Py4J calls.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Manually tested as demonstrated in https://github.com/apache/spark/pull/49312

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #49313 from HyukjinKwon/SPARK-50685.

Authored-by: Hyukjin Kwon <gurwls223@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
H
Hyukjin Kwon committed
2d320aaa420feeb646def7f5aeffaa02fa68508e
Parent: 9297c5d