mirror of
https://github.com/apache/arrow.git
synced 2026-04-06 06:17:29 +00:00
main
766 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Sarah Gilmore
|
df88383ba1 |
GH-49611: [MATLAB] MATLAB workflow failing due to action permission error (#49650)
### Rationale for this change The MATLAB workflow began (silently) failing on March 20th with this startup error: > The actions matlab-actions/setup-matlab@ v2 and matlab-actions/run-tests@ v2 are not allowed in apache/arrow because all actions must be from a repository owned by your enterprise, created by GitHub, or match one of the patterns: ... The MATLAB workflows began failing in CI because it depends on two actions, `matlab-actions/setup-matlab` and `matlab-actions/run-tests`, that, prior to yesterday, were not included in [apache/infrastructure-actions](https://github.com/apache/infrastructure-actions) "action" allow list. Both `matlab-actions/setup-matlab` and `matlab-actions/run-tests` are now included in the action allow list. See [PR#6423](https://github.com/apache/infrastructure-actions/pull/643) and [PR#644](https://github.com/apache/infrastructure-actions/pull/644) for details. ### Component(s) MATLAB, Continuous Integration ### What changes are included in this PR? 1. Pinned `matlab-actions/setup-matlab` to version `v2.7.0`, specified by git hash `aa8bbc7b76daa63c5d456d1430cbd6cb5b626ab4`. 2. Pinned `matlab-actions/run-tests` to version `v2.3.1`, specified by git hash `4a3d2e8bdc811f72defb8122e46a009312acc198`. ### Are these changes tested? N/A ### Are there any user-facing changes? No. * GitHub Issue: #49611 Lead-authored-by: Sarah Gilmore <sgilmore@mathworks.com> Co-authored-by: Sarah Gilmore <74676073+sgilmore10@users.noreply.github.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Sarah Gilmore <sgilmore@mathworks.com> |
||
Alina (Xi) Li
|
d5e58658ca |
GH-49463: [C++][FlightRPC] Add Ubuntu ODBC Support (#49564)
### Rationale for this change GH-49463 Add Ubuntu support so users can connect using odbc on Linux. ### What changes are included in this PR? - Enable Linux ODBC build with unicode support - Add ODBC Ubuntu build in CI - Added `docker-compose` for Flight SQL ODBC - Register ODBC after build - Replaced `boost::lexicographical_compare` with `std::lexicographical_compare` - Fixed conversion bugs in `SetAttributeSQLWCHAR` - Convert from std::string to std::u16string directly without involving wide string (wstring) - Enabling ODBC Linux test build will be added in a separate PR ### Are these changes tested? - Ubuntu build is tested in CI - ODBC is tested with `isql` on local docker, and confirmed to be able to connect to online remote instance ### Are there any user-facing changes? N/A * GitHub Issue: #49463 Lead-authored-by: Alina (Xi) Li <alinal@bitquilltech.com> Co-authored-by: Alina (Xi) Li <alina.li@improving.com> Co-authored-by: Alina (Xi) Li <96995091+alinaliBQ@users.noreply.github.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
Raúl Cumplido
|
560ef02158 |
GH-46600: [C++][CI] Add job with ARROW_LARGE_MEMORY_TESTS enabled (#49490)
### Rationale for this change Now that we have self-hosted runners with AWS we should test the `ARROW_LARGE_MEMORY_TESTS` on CI. ### What changes are included in this PR? Added new runner for ARROW_LARGE_MEMORY_TESTS. Fix `parquet-writer-test` to generate huge expected page using huge `max_rows_per_page` instead of default. ### Are these changes tested? Yes via CI ### Are there any user-facing changes? No * GitHub Issue: #46600 Authored-by: Raúl Cumplido <raulcumplido@gmail.com> Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com> |
||
Sutou Kouhei
|
10eaafd2b4 |
GH-49544: [Ruby] Add benchmark for readers (#49545)
### Rationale for this change
Performance is important in Apache Arrow. So benchmark is useful for developing Apache Arrow implementation.
### What changes are included in this PR?
* Add benchmarks for file and streaming readers.
* Add support for `mmap` in streaming reader.
Here are benchmark results on my environment.
Pure Ruby implementation is about 5-6x slower than release build C++ implementation but a bit faster than debug build C++ implementation.
Release build C++/GLib:
File format:
```console
$ ruby -v -S benchmark-driver ruby/red-arrow-format/benchmark/file-reader.yaml
ruby 4.1.0dev (2026-02-19T09:04:23Z master 6bb0b6b16c) +PRISM [x86_64-linux]
Warming up --------------------------------------
Arrow::Table.load 11.207k i/s - 12.188k times in 1.087487s (89.23μs/i)
Arrow::RecordBatchFileReader 19.724k i/s - 21.296k times in 1.079727s (50.70μs/i)
ArrowFormat::FileReader 3.555k i/s - 3.883k times in 1.092223s (281.28μs/i)
Calculating -------------------------------------
Arrow::Table.load 11.483k i/s - 33.622k times in 2.928024s (87.09μs/i)
Arrow::RecordBatchFileReader 19.673k i/s - 59.170k times in 3.007729s (50.83μs/i)
ArrowFormat::FileReader 3.574k i/s - 10.665k times in 2.984214s (279.81μs/i)
Comparison:
Arrow::RecordBatchFileReader: 19672.6 i/s
Arrow::Table.load: 11482.8 i/s - 1.71x slower
ArrowFormat::FileReader: 3573.8 i/s - 5.50x slower
```
Streaming format:
```console
$ ruby -v -S benchmark-driver ruby/red-arrow-format/benchmark/streaming-reader.yaml
ruby 4.1.0dev (2026-02-19T09:04:23Z master 6bb0b6b16c) +PRISM [x86_64-linux]
Warming up --------------------------------------
Arrow::Table.load 11.360k i/s - 12.485k times in 1.099067s (88.03μs/i)
Arrow::RecordBatchStreamReader 20.180k i/s - 21.857k times in 1.083126s (49.56μs/i)
ArrowFormat::StreamingReader 3.398k i/s - 3.400k times in 1.000479s (294.26μs/i)
Calculating -------------------------------------
Arrow::Table.load 11.397k i/s - 34.078k times in 2.990170s (87.74μs/i)
Arrow::RecordBatchStreamReader 20.039k i/s - 60.538k times in 3.020964s (49.90μs/i)
ArrowFormat::StreamingReader 3.340k i/s - 10.195k times in 3.052059s (299.37μs/i)
Comparison:
Arrow::RecordBatchStreamReader: 20039.3 i/s
Arrow::Table.load: 11396.7 i/s - 1.76x slower
ArrowFormat::StreamingReader: 3340.4 i/s - 6.00x slower
```
Debug build C++/GLib:
File format:
```console
$ ruby -v -S benchmark-driver ruby/red-arrow-format/benchmark/file-reader.yaml
ruby 4.1.0dev (2026-02-19T09:04:23Z master 6bb0b6b16c) +PRISM [x86_64-linux]
Warming up --------------------------------------
Arrow::Table.load 2.175k i/s - 2.200k times in 1.011375s (459.72μs/i)
Arrow::RecordBatchFileReader 3.129k i/s - 3.421k times in 1.093397s (319.61μs/i)
ArrowFormat::FileReader 3.384k i/s - 3.430k times in 1.013625s (295.52μs/i)
Calculating -------------------------------------
Arrow::Table.load 2.145k i/s - 6.525k times in 3.041760s (466.17μs/i)
Arrow::RecordBatchFileReader 3.020k i/s - 9.386k times in 3.108456s (331.18μs/i)
ArrowFormat::FileReader 3.368k i/s - 10.151k times in 3.013576s (296.87μs/i)
Comparison:
ArrowFormat::FileReader: 3368.4 i/s
Arrow::RecordBatchFileReader: 3019.5 i/s - 1.12x slower
Arrow::Table.load: 2145.1 i/s - 1.57x slower
```
Streaming format:
```console
$ ruby -v -S benchmark-driver ruby/red-arrow-format/benchmark/streaming-reader.yaml
ruby 4.1.0dev (2026-02-19T09:04:23Z master 6bb0b6b16c) +PRISM [x86_64-linux]
Warming up --------------------------------------
Arrow::Table.load 2.115k i/s - 2.140k times in 1.011815s (472.81μs/i)
Arrow::RecordBatchStreamReader 3.052k i/s - 3.355k times in 1.099273s (327.65μs/i)
ArrowFormat::StreamingReader 3.283k i/s - 3.290k times in 1.002016s (304.56μs/i)
Calculating -------------------------------------
Arrow::Table.load 2.198k i/s - 6.345k times in 2.886603s (454.94μs/i)
Arrow::RecordBatchStreamReader 3.105k i/s - 9.156k times in 2.948523s (322.03μs/i)
ArrowFormat::StreamingReader 3.225k i/s - 9.850k times in 3.054339s (310.09μs/i)
Comparison:
ArrowFormat::StreamingReader: 3224.9 i/s
Arrow::RecordBatchStreamReader: 3105.3 i/s - 1.04x slower
Arrow::Table.load: 2198.1 i/s - 1.47x slower
```
### Are these changes tested?
Yes.
### Are there any user-facing changes?
No.
* GitHub Issue: #49544
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
|
||
justing-bq
|
21a2d4f81d |
GH-49268: [C++][FlightRPC] Fix ODBC tests for MacOS (#49267)
### Rationale for this change Addresses https://github.com/apache/arrow/issues/49268 ### What changes are included in this PR? - Fixed Mock Server setup to work on both Windows and MacOS. - Changed Mocker Server setup logic to happen once at the Test Environment level. - Changed test connect/disconnect logic to happen once per test fixture instead of once per test. - Disabled tests that are not appliable to MacOS (usually because iODBC doesn't support the relevant functionality). - Adjusted expected SQL States where different on Mac. - Fixed any tests failing on MacOS. ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: #49268 Lead-authored-by: Alina (Xi) Li <alinal@bitquilltech.com> Co-authored-by: justing-bq <62349012+justing-bq@users.noreply.github.com> Co-authored-by: Alina (Xi) Li <96995091+alinaliBQ@users.noreply.github.com> Co-authored-by: justing-bq <justin.gossett@improving.com> Co-authored-by: alinalibq <alina.li@improving.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
tadeja
|
771205b004 |
GH-49435: [CI][C++] Fix macOS build with Homebrew protobuf v34 (#49491)
### Rationale for this change Closes #49435 ### What changes are included in this PR? a) Add brew update to `cpp.yml` and `python.yml` for GitHub Actions runner to use updated formulae b) Handle 'nodiscard' return values in Flight and Substrait See https://github.com/grpc/grpc/issues/41755 (See also previous discussion in [#49436](https://github.com/apache/arrow/pull/49436#issuecomment-4003351021)) ### Are these changes tested? Yes, macOS C++ and Python CI jobs pass. ### Are there any user-facing changes? No. * GitHub Issue: #49435 Lead-authored-by: Tadeja Kadunc <tadeja.kadunc@gmail.com> Co-authored-by: tadeja <tadeja@users.noreply.github.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com> |
||
dependabot[bot]
|
b69ce86617 |
MINOR: [CI] Bump docker/login-action from 3.7.0 to 4.0.0 (#49479)
Bumps [docker/login-action](https://github.com/docker/login-action) from 3.7.0 to 4.0.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/docker/login-action/releases">docker/login-action's releases</a>.</em></p> <blockquote> <h2>v4.0.0</h2> <ul> <li>Node 24 as default runtime (requires <a href="https://github.com/actions/runner/releases/tag/v2.327.1">Actions Runner v2.327.1</a> or later) by <a href="https://github.com/crazy-max"><code>@crazy-max</code></a> in <a href="https://redirect.github.com/docker/login-action/pull/929">docker/login-action#929</a></li> <li>Switch to ESM and update config/test wiring by <a href="https://github.com/crazy-max"><code>@crazy-max</code></a> in <a href="https://redirect.github.com/docker/login-action/pull/927">docker/login-action#927</a></li> <li>Bump <code>@actions/core</code> from 1.11.1 to 3.0.0 in <a href="https://redirect.github.com/docker/login-action/pull/919">docker/login-action#919</a></li> <li>Bump <code>@aws-sdk/client-ecr</code> from 3.890.0 to 3.1000.0 in <a href="https://redirect.github.com/docker/login-action/pull/909">docker/login-action#909</a> <a href="https://redirect.github.com/docker/login-action/pull/920">docker/login-action#920</a></li> <li>Bump <code>@aws-sdk/client-ecr-public</code> from 3.890.0 to 3.1000.0 in <a href="https://redirect.github.com/docker/login-action/pull/909">docker/login-action#909</a> <a href="https://redirect.github.com/docker/login-action/pull/920">docker/login-action#920</a></li> <li>Bump <code>@docker/actions-toolkit</code> from 0.63.0 to 0.77.0 in <a href="https://redirect.github.com/docker/login-action/pull/910">docker/login-action#910</a> <a href="https://redirect.github.com/docker/login-action/pull/928">docker/login-action#928</a></li> <li>Bump <code>@isaacs/brace-expansion</code> from 5.0.0 to 5.0.1 in <a href="https://redirect.github.com/docker/login-action/pull/921">docker/login-action#921</a></li> <li>Bump js-yaml from 4.1.0 to 4.1.1 in <a href="https://redirect.github.com/docker/login-action/pull/901">docker/login-action#901</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/docker/login-action/compare/v3.7.0...v4.0.0">https://github.com/docker/login-action/compare/v3.7.0...v4.0.0</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
Sutou Kouhei
|
ed37b14f40 |
GH-49422: [CI][Integration][Ruby] Add the Ruby implementation (#49423)
### Rationale for this change There are some missing features in the Ruby implmentaion for now but we can pass them by skipping some tests in our integration tests. ### What changes are included in this PR? Archery: * Add `--with-ruby` to `archery integration` * Add `archery.integration.tester_ruby.RubyTester` * Add `no_map_field_names_validate` quirk for GH-49415 * Show environment variables too on external command failure because Ruby tester uses environment variables not command line arguments to pass information to integration tester * Use `ARCHERY_INTEGRATION_WITH_CPP=1` instead of `ARROW_INTEGRATION_CPP=ON` like other implementations such as `ARCHERY_INTEGRATION_WITH_GO` Ruby: * Add `red-arrow-format-integration-test` as the test driver * This is not included in `.gem` because this is only for development * Add `ruby/red-arrow-format/lib/arrow-format/integration/` as helpers of the test driver * This is not included in `.gem` because this is only for development * Add `ArrowFormat::Array#empty?` * Add `ArrowFormat::RecordBatch#empty?` * Add `ArrowFormat::NullArray#n_nulls` * `ArrowFormat::*Array#to_a`: Add support for empty case * Fix Apache Arrow decimal <-> `BigDecimal` conversion * `ArrowFormat::Bitmap#each`: Fix a bug that one bit is ignored * Move dictionary ID to `ArrowFormat::DictionaryType` from `ArrowFormat::Field` * Add support for V4 union that has validity bitmap * Add support for no continuation token message for backward compatibility * `ArrowFormat::StreamingReader`: Add support for reading schema without calling `#each` * `ArrowFormat::MapType`: Add support for keys sorted * `ArrowFormat::MapType`: Always use "key"/"value"/"entries" for field names ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes. * GitHub Issue: #49422 Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
|
Raúl Cumplido
|
f1b21f1945 |
GH-36411: [Python] Use scikit-build-core as build backend for PyArrow and get rid of setup.py (#49259)
### Rationale for this change Move our PyArrow build backend from setuptools and a custom setup.py to scikit-build-core which is just build backend for CMake related projects. ### What changes are included in this PR? Move from setuptools to scikit-build-core and remove PyArrow setup.py. Update some of the build requirements and minor fixes. A custom build backend has been also been created in order to wrap scikit-build-core in order to fix problems on License files for monorepos. pyproject.toml metadata validation expects license files to exist before exercising the build backend that's why we create symlinks. Our thin build backend will just make those symlinks hard-links in order for license and notice files to contain the contents and be added as part of the sdist. Remove flags that are not used anymore (were only part of setup.py) and documented and validated how the same flags have to be used now. ### Are these changes tested? Yes all Python CI tests, wheels and sdist are successful. ### Are there any user-facing changes? Yes, users building PyArrow will now require the new build dependencies to exercise the build and depending on the flags used they might require to use the new documented way of using those flags. * GitHub Issue: #36411 Lead-authored-by: Raúl Cumplido <raulcumplido@gmail.com> Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com> Co-authored-by: Rok Mihevc <rok@mihevc.org> Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com> |
||
|
justing-bq
|
bbf337b83a |
GH-49283: [C++][FlightRPC] Add separate release & debug workflows for MacOS ODBC (#49284)
### Rationale for this change Addresses https://github.com/apache/arrow/issues/49283 ### What changes are included in this PR The ODBC MacOS workflows now have release and debug workflows. ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: #49283 Lead-authored-by: justing-bq <62349012+justing-bq@users.noreply.github.com> Co-authored-by: Alina (Xi) Li <alinal@bitquilltech.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
|
Alina (Xi) Li
|
8e625d0a34 |
GH-49219: [C++][FlightRPC] Enable static ODBC build on macOS (#49220)
### Rationale for this change #49219 Support static ODBC build on macOS. Linux support will be in a different PR. ### What changes are included in this PR? - Add unix-specific CMake build commands in CMakeLists.txt to link ODBC DYLIB statically to dependencies. - Header file changes due to switch to static build. - Test file changes due to static macOS linking that caused different behavior in iodbc ### Are these changes tested? Build is tested in CI. ### Are there any user-facing changes? N/A * GitHub Issue: #49219 Lead-authored-by: Alina (Xi) Li <alinal@bitquilltech.com> Co-authored-by: Alina (Xi) Li <alina.li@improving.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
Rok Mihevc
|
c48e81e4d0 |
GH-48593: [C++] C++20: use standard calendar / timezone APIs (#48601)
### Rationale for this change Switch to std::chrono for MSVC to be able to use the system-provided timezone automatically on Windows. ### What changes are included in this PR? This adds `chrono_internal.h` that uses C++20 std::chrono timezone/calendar APIs on compilers with support (MSVC only for now) and falls back to vendored `date.h` otherwise. ### Are these changes tested? Partially tested locally and partially to be tested on CI. ### Are there any user-facing changes? Yes, Windows users will no longer need to install the IANA tzdb (see [instructions here](https://arrow.apache.org/docs/cpp/build_system.html#runtime-dependencies) and [here](https://arrow.apache.org/docs/python/install.html#tzdata-on-windows)). We possibly have tzdb download set up in CI too and should update it appropriately. * GitHub Issue: #48593 Authored-by: Rok Mihevc <rok@mihevc.org> Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com> |
||
|
dependabot[bot]
|
3ff230c534 |
MINOR: [CI] Bump actions/download-artifact from 7 to 8 (#49431)
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 7 to 8. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/actions/download-artifact/releases">actions/download-artifact's releases</a>.</em></p> <blockquote> <h2>v8.0.0</h2> <h2>v8 - What's new</h2> <h3>Direct downloads</h3> <p>To support direct uploads in <code>actions/upload-artifact</code>, the action will no longer attempt to unzip all downloaded files. Instead, the action checks the <code>Content-Type</code> header ahead of unzipping and skips non-zipped files. Callers wishing to download a zipped file as-is can also set the new <code>skip-decompress</code> parameter to <code>false</code>.</p> <h3>Enforced checks (breaking)</h3> <p>A previous release introduced digest checks on the download. If a download hash didn't match the expected hash from the server, the action would log a warning. Callers can now configure the behavior on mismatch with the <code>digest-mismatch</code> parameter. To be secure by default, we are now defaulting the behavior to <code>error</code> which will fail the workflow run.</p> <h3>ESM</h3> <p>To support new versions of the @ actions/* packages, we've upgraded the package to ESM.</p> <h2>What's Changed</h2> <ul> <li>Don't attempt to un-zip non-zipped downloads by <a href="https://github.com/danwkennedy"><code>@danwkennedy</code></a> in <a href="https://redirect.github.com/actions/download-artifact/pull/460">actions/download-artifact#460</a></li> <li>Add a setting to specify what to do on hash mismatch and default it to <code>error</code> by <a href="https://github.com/danwkennedy"><code>@danwkennedy</code></a> in <a href="https://redirect.github.com/actions/download-artifact/pull/461">actions/download-artifact#461</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/download-artifact/compare/v7...v8.0.0">https://github.com/actions/download-artifact/compare/v7...v8.0.0</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
693bcfec14 |
MINOR: [CI] Bump actions/upload-artifact from 6 to 7 (#49430)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 6 to 7. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/actions/upload-artifact/releases">actions/upload-artifact's releases</a>.</em></p> <blockquote> <h2>v7.0.0</h2> <h2>v7 What's new</h2> <h3>Direct Uploads</h3> <p>Adds support for uploading single files directly (unzipped). Callers can set the new <code>archive</code> parameter to <code>false</code> to skip zipping the file during upload. Right now, we only support single files. The action will fail if the glob passed resolves to multiple files. The <code>name</code> parameter is also ignored with this setting. Instead, the name of the artifact will be the name of the uploaded file.</p> <h3>ESM</h3> <p>To support new versions of the <code>@actions/*</code> packages, we've upgraded the package to ESM.</p> <h2>What's Changed</h2> <ul> <li>Add proxy integration test by <a href="https://github.com/Link"><code>@Link</code></a>- in <a href="https://redirect.github.com/actions/upload-artifact/pull/754">actions/upload-artifact#754</a></li> <li>Upgrade the module to ESM and bump dependencies by <a href="https://github.com/danwkennedy"><code>@danwkennedy</code></a> in <a href="https://redirect.github.com/actions/upload-artifact/pull/762">actions/upload-artifact#762</a></li> <li>Support direct file uploads by <a href="https://github.com/danwkennedy"><code>@danwkennedy</code></a> in <a href="https://redirect.github.com/actions/upload-artifact/pull/764">actions/upload-artifact#764</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/Link"><code>@Link</code></a>- made their first contribution in <a href="https://redirect.github.com/actions/upload-artifact/pull/754">actions/upload-artifact#754</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/upload-artifact/compare/v6...v7.0.0">https://github.com/actions/upload-artifact/compare/v6...v7.0.0</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
Sutou Kouhei
|
f3f1eb0f1f |
GH-49341: [Packaging] Add support for Ubuntu 26.04 (#49345)
### Rationale for this change Ubuntu 26.04 will be the next LTS for Ubuntu. It'll be released on 2026-04. ### What changes are included in this PR? * Add `Dockerfile`s for Ubuntu 26.04 * Add Ubuntu 26.04 entries * Add a labeler configuration for Linux packages ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes. * GitHub Issue: #49341 Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com> |
||
|
Alina (Xi) Li
|
e2aad46a07 |
GH-48576: [C++][FlightRPC] ODBC: add Mac setup script (#48578)
### Rationale for this change #48576 ### What changes are included in this PR? - Added Mac Setup ODBC ini Script ### Are these changes tested? Script is tested in CI. Tested locally on macOS. ### Are there any user-facing changes? N/A * GitHub Issue: #48576 Lead-authored-by: Alina (Xi) Li <alinal@bitquilltech.com> Co-authored-by: Victor Tsang <victor.tsang@improving.com> Co-authored-by: Alina (Xi) Li <alina.li@improving.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
|
tadeja
|
9ed8fc1ed2 |
GH-49184: [CI] AMD64 macOS 15-intel Python 3 consistently times out (#49189)
### Rationale for this change Recent CI checks failing with the job `AMD64 macOS 15-intel Python 3` being cancelled at 60 minutes. ```The job has exceeded the maximum execution time of 1h0m0s``` ### What changes are included in this PR? Disabling large memory tests for macOS 15-intel only. For both macOS 14 and 15 adding PYTEST_ARGS: "-n auto --durations=40" to run tests across multiple CPUs (workers) and also output slowest 40 durations. ### Are these changes tested? Tested on CI. ### Are there any user-facing changes? No. * GitHub Issue: #49184 Authored-by: Tadeja Kadunc <tadeja.kadunc@gmail.com> Signed-off-by: Rok Mihevc <rok@mihevc.org> |
||
|
Raúl Cumplido
|
4b19bf0f04 |
GH-49084: [CI][Dev] Wait for odbc-nightly before executing CPP extra report job (#49085)
### Rationale for this change The report is currently shown as failed because we are not waiting for the nightly job to finish. ### What changes are included in this PR? Add `odbc-nightly` to the list of needed jobs before running report. ### Are these changes tested? No, but really minor ### Are there any user-facing changes? No * GitHub Issue: #49084 Authored-by: Raúl Cumplido <raulcumplido@gmail.com> Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com> |
||
|
Rok Mihevc
|
a444c87a6e |
MINOR: [CI] Set max-parallel=20 (#49228)
### Rationale for this change We want to comply with [ASF policy](https://infra.apache.org/github-actions-policy.html) ### What changes are included in this PR? Sets `max-parallel=20` in .github/workflows/python.yml ### Are these changes tested? No. ### Are there any user-facing changes? Only to CI users. Authored-by: Rok Mihevc <rok@mihevc.org> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
|
Alina (Xi) Li
|
f32f8ac2ee |
GH-48904: [C++][FlightRPC][CI][Packaging] Upload ODBC installer into GitHub release as RC (#48934)
### Rationale for this change #48904 Upload ODBC as a GitHub draft release upon release candidate tag ### What changes are included in this PR? - Create a draft GitHub release for ODBC, and upload the ODBC MSI to the draft release. ODBC release is only triggered by RC tag - add gh release ODBC download pattern to `04-binary-download.sh` - add gh release ODBC upload pattern to `05-binary-upload.sh` ### Are these changes tested? - CI changes tested in forked repository, a draft GitHub release is created - `04-binary-download.sh` and `05-binary-upload.sh` changes are not tested ### Are there any user-facing changes? Yes, this PR adds GitHub release for Apache Arrow Flight SQL ODBC MSI installer. * GitHub Issue: #48904 Lead-authored-by: Alina (Xi) Li <alina.li@improving.com> Co-authored-by: Alina (Xi) Li <alinal@bitquilltech.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
|
dependabot[bot]
|
649e1b67f4 |
MINOR: [CI] Bump docker/login-action from 3.6.0 to 3.7.0 (#49191)
Bumps [docker/login-action](https://github.com/docker/login-action) from 3.6.0 to 3.7.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/docker/login-action/releases">docker/login-action's releases</a>.</em></p> <blockquote> <h2>v3.7.0</h2> <ul> <li>Add <code>scope</code> input to set scopes for the authentication token by <a href="https://github.com/crazy-max"><code>@crazy-max</code></a> in <a href="https://redirect.github.com/docker/login-action/pull/912">docker/login-action#912</a></li> <li>Add support for AWS European Sovereign Cloud ECR by <a href="https://github.com/dphi"><code>@dphi</code></a> in <a href="https://redirect.github.com/docker/login-action/pull/914">docker/login-action#914</a></li> <li>Ensure passwords are redacted with <code>registry-auth</code> input by <a href="https://github.com/crazy-max"><code>@crazy-max</code></a> in <a href="https://redirect.github.com/docker/login-action/pull/911">docker/login-action#911</a></li> <li>build(deps): bump lodash from 4.17.21 to 4.17.23 in <a href="https://redirect.github.com/docker/login-action/pull/915">docker/login-action#915</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/docker/login-action/compare/v3.6.0...v3.7.0">https://github.com/docker/login-action/compare/v3.6.0...v3.7.0</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
Alina (Xi) Li
|
02d05f4267 |
GH-49190: [C++][CI] Fix unknown job 'odbc' error in C++ Extra Workflow (#49192)
### Rationale for this change See #49190 ### What changes are included in this PR? Fix `unknown job 'odbc' error` caused by typo ### Are these changes tested? Tested in CI ### Are there any user-facing changes? N/A * GitHub Issue: #49190 Authored-by: Alina (Xi) Li <alinal@bitquilltech.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
|
Alina (Xi) Li
|
d5fa7cb610 |
GH-48575: [C++][FlightRPC] Standalone ODBC macOS CI (#48577)
### Rationale for this change #48575 ### What changes are included in this PR? - Add new ODBC workflow for macOS Intel 15 and 14 arm64. - Added ODBC build fixes to enable build on macOS CI. ### Are these changes tested? Tested in CI and local macOS Intel and M1 environments. ### Are there any user-facing changes? N/A * GitHub Issue: #48575 Lead-authored-by: Alina (Xi) Li <alina.li@improving.com> Co-authored-by: justing-bq <62349012+justing-bq@users.noreply.github.com> Co-authored-by: Victor Tsang <victor.tsang@improving.com> Co-authored-by: Alina (Xi) Li <alinal@bitquilltech.com> Co-authored-by: vic-tsang <victor.tsang@improving.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
|
Alina (Xi) Li
|
654fb284f9 |
GH-49092: [C++][FlightRPC][CI] Nightly Packaging: Add dev-yyyy-mm-dd to ODBC MSI name (#49151)
### Rationale for this change #49092 ### What changes are included in this PR? - Add `dev-yyyy-mm-dd` to ODBC MSI name. This is a similar approach to R nightly. Before: `Apache Arrow Flight SQL ODBC-1.0.0-win64.msi`. After: `Apache Arrow Flight SQL ODBC-1.0.0-dev-2026-02-04-win64.msi`. ### Are these changes tested? Tested in CI. Successfully renamed file: https://github.com/apache/arrow/actions/runs/21686252848/job/62534629714?pr=49151#step:3:26 ### Are there any user-facing changes? Yes, the nightly ODBC file names will be changed as described above. * GitHub Issue: #49092 Authored-by: Alina (Xi) Li <alina.li@improving.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
|
Alina (Xi) Li
|
baba1d4b59 |
GH-49004: [C++][FlightRPC] Run ODBC tests in workflow using cpp_test.sh (#49005)
### Rationale for this change #49004 ### What changes are included in this PR? - Run tests using `cpp_test.sh` in the ODBC job of C++ Extra CI. Note: `find_package(Arrow)` check in `cpp_test.sh` is disabled due to blocker GH-49050 ### Are these changes tested? Yes, in CI ### Are there any user-facing changes? N/A * GitHub Issue: #49004 Lead-authored-by: Alina (Xi) Li <alina.li@improving.com> Co-authored-by: Alina (Xi) Li <96995091+alinaliBQ@users.noreply.github.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
Nate Prewitt
|
ef5854a868 |
GH-41990: [C++] Fix AzureFileSystem compilation on Windows (#48971)
Let me preface this pull request that I have not worked in C++ in quite a while. Apologies if this is missing modern idioms or is an obtuse fix. ### Rationale for this change I encountered an issue trying to compile the AzureFileSystem backend in C++ on Windows. Searching the issue tracker, it appears this is already a [known](https://github.com/apache/arrow/issues/41990) but unresolved problem. This is an attempt to either address the issue or move the conversation forward for someone more experienced. ### What changes are included in this PR? AzureFileSystem uses `unique_ptr` while the other cloud file system implementations rely on `shared_ptr`. Since this is a forward-declared Impl in the headers file but the destructor was defined inline (via `= default`), we're getting compilation issues with MSVC due to it requiring the complete type earlier than GCC/Clang. This change removes the defaulted definition from the header file and moves it into the .cc file where we have a complete type. Unrelated, I've also wrapped 2 exception variables in `ARROW_UNUSED`. These are warnings treated as errors by MSVC at compile time. This was revealed in CI after resolving the issue above. ### Are these changes tested? I've enabled building and running the test suite in GHA in 8dd62d62a9af022813e9c8662956740340a9473f. I believe a large portion of those tests may be skipped though since Azurite isn't present from what I can see. I'm not tied to the GHA updates being included in the PR, it's currently here for demonstration purposes. I noticed the other FS implementations are also not built and tested on Windows. One quirk of this PR is getting WIL in place to compile the Azure C++ SDK was not intuitive for me. I've placed a dummy `wilConfig.cmake` to get the Azure SDK to build, but I'd assume there's a better way to do this. I'm happy to refine the build setup if we choose to keep it. ### Are there any user-facing changes? Nothing here should affect user-facing code beyond fixing the compilation issues. If there are concerns for things I'm missing, I'm happy to discuss those. * GitHub Issue: #41990 Lead-authored-by: Nate Prewitt <nateprewitt@microsoft.com> Co-authored-by: Nate Prewitt <nate.prewitt@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Co-authored-by: Antoine Pitrou <pitrou@free.fr> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
|
Rok Mihevc
|
1cd1841c06 |
GH-49150: [Doc][CI][Python] Doctests failing on rst files due to pandas 3+ (#49088)
Fixes: #49150 See https://github.com/apache/arrow/pull/48619#issuecomment-3823269381 ### Rationale for this change Fix CI failures ### What changes are included in this PR? Tests are made more general to allow for Pandas 2 and Pandas 3 style string types ### Are these changes tested? By CI ### Are there any user-facing changes? No * GitHub Issue: #49150 Authored-by: Rok Mihevc <rok@mihevc.org> Signed-off-by: Rok Mihevc <rok@mihevc.org> |
||
Nic Crane
|
f8955125c9 |
MINOR: [Docs] Add links to AI-generated code guidance (#49131)
### Rationale for this change Add link to AI-generated code guidance - we should make sure the docs are updated before we merge this though ### What changes are included in this PR? Add link to AI-generated code guidance ### Are these changes tested? No ### Are there any user-facing changes? No Lead-authored-by: Nic Crane <thisisnic@gmail.com> Co-authored-by: Raúl Cumplido <raulcumplido@gmail.com> Signed-off-by: Nic Crane <thisisnic@gmail.com> |
||
|
Alina (Xi) Li
|
5a71d2a404 |
GH-48932: [C++][Packaging][FlightRPC] Fix rsync build error ODBC Nightly Package (#48933)
### Rationale for this change #48932 ### What changes are included in this PR? - Fix `rsync` build error ODBC Nightly Package ### Are these changes tested? - tested in CI ### Are there any user-facing changes? - After fix, users should be able to get Nightly ODBC package release * GitHub Issue: #48932 Authored-by: Alina (Xi) Li <alina.li@improving.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
|
Alina (Xi) Li
|
894d6a3d53 |
GH-48872: [C++][FlightRPC][CI][Packaging] Upload ODBC to Nightly Release (#48873)
### Rationale for this change https://github.com/apache/arrow/issues/48872 ### What changes are included in this PR? - Upload Flight SQL ODBC installer msi file to Nightly release `https://nightlies.apache.org/arrow/odbc/` ### Are these changes tested? since we cannot use secrets in PRs, the changes will take in effect after merge ### Are there any user-facing changes? Yes, Apache Arrow Flight SQL ODBC installer msi file will be uploaded to Nightly releases. GitHub release will be in a separate PR. * GitHub Issue: #48872 Authored-by: Alina (Xi) Li <alina.li@improving.com> Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com> |
||
|
Raúl Cumplido
|
e4c9ed2989 |
GH-48884: [Dev][Release] Remove non-published draft release candidates when publishing official release to GitHub (#48887)
### Rationale for this change
Draft releases appear on top of the releases page on GitHub, cluttering the releases page.
### What changes are included in this PR?
Once we publish the official release we can delete draft release candidates. This can be done automatically on the `release.yml` workflow.
### Are these changes tested?
Not on the workflow but I've deleted the draft releases for 23.0.0 (RC0 and RC1) executing the same command:
```
$ gh release list --jq '.[] | select(.isDraft).tagName' --json isDraft,tagName --repo apache/arrow
apache-arrow-23.0.0-rc1
apache-arrow-23.0.0-rc0
$ gh release list --jq '.[] | select(.isDraft).tagName' --json isDraft,tagName --repo apache/arrow | while read tag; do gh release delete ${tag} --repo apache/arrow; done
✓ Deleted release apache-arrow-23.0.0-rc1
✓ Deleted release apache-arrow-23.0.0-rc0
```
### Are there any user-facing changes?
No
* GitHub Issue: #48884
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
|
||
Bryce Mecum
|
ddcf2dcbe9 |
MINOR: [CI][Release] Fix incorrect path in release_candidate.yml (#48871)
### Rationale for this change I noticed a reference to a `release_candidate.sh` in the `paths` field in `release_candidate.yml` which is a file that doesn't exist. I think this was just a typo made during refactoring. ### What changes are included in this PR? Corrected `paths` list entry. ### Are these changes tested? No. ### Are there any user-facing changes? No. Authored-by: Bryce Mecum <petridish@gmail.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
Antoine Prouvost
|
a1ec5a9a1d |
GH-47167: [C++][Dev] Update clang-format dependency (#47168)
### Rationale for this change Update clang-format to better match modern IDEs. ### What changes are included in this PR? - Update clang-format - Reformat code base with newer version ### Are these changes tested? Yes ### Are there any user-facing changes? No * GitHub Issue: #47167 Lead-authored-by: AntoinePrv <AntoinePrv@users.noreply.github.com> Co-authored-by: Rossi Sun <zanmato1984@gmail.com> Signed-off-by: Rossi Sun <zanmato1984@gmail.com> |
||
|
Raúl Cumplido
|
4f77a06e1f |
GH-48841: [Release][Package] Add GH_TOKEN to rake build step on Linux Packaging jobs (#48842)
### Rationale for this change With: - https://github.com/apache/arrow/pull/48839 We use `gh release download`. This requires the GH_TOKEN available. ### What changes are included in this PR? Add env with `GH_TOKEN`. I've validate the Rake's `sh` should inherit the environment variables that are defined on your shell. ### Are these changes tested? No ### Are there any user-facing changes? No * GitHub Issue: #48841 Authored-by: Raúl Cumplido <raulcumplido@gmail.com> Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com> |
||
Jacob Wujciak-Jens
|
f82dede3b2 |
GH-44817: [CI] Comment new repo url on issues of moved components (#44818)
### Rationale for this change Point people to the correct repo. ### What changes are included in this PR? - Add a comment with link to new repo - Do not add Component labels to issue ### Are these changes tested? I tested it on my fork: https://github.com/assignUser/arrow/issues/23 ### Are there any user-facing changes? No * GitHub Issue: #44817 Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Co-authored-by: Nic Crane <thisisnic@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Nic Crane <thisisnic@gmail.com> |
||
|
Sutou Kouhei
|
288cd43bdc |
GH-48623: [CI][Archery][Dev] Add missing headers to email reports (#48624)
### Rationale for this change
Our email reports miss the following headers:
* `MIME-Version: 1.0`
* `Content-Type: text/plain; charset="utf-8"`
* `Message-Id: ${AUTO_GENERATED_MESSAGE_ID}`
* `Date: ${DATE_IN_RFC_2822}`
### What changes are included in this PR?
Add these headers.
### Are these changes tested?
Yes.
### Are there any user-facing changes?
No.
* GitHub Issue: #48623
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
|
||
|
Alina (Xi) Li
|
cb100b2078 |
GH-48594: [C++][FlightRPC] Fix ODBC CI Long Build Time Issue (#48595)
### Rationale for this change #48594 ### What changes are included in this PR? - Resolve caching issue inside ODBC CI, so build time is generally less than 1hr - Removed environment variables that are not necessary for ODBC building to keep workflow simple - Changed ODBC workflow to be triggered when ODBC files are changed - Add `CI: Extra: C++` label when ODBC files pr C++ Extra workflow is changed ### Are these changes tested? - Tested in CI ### Are there any user-facing changes? N/A * GitHub Issue: #48594 Authored-by: Alina (Xi) Li <alina.li@improving.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
|
dependabot[bot]
|
4e4026c9b9 |
MINOR: [CI] Bump actions/upload-artifact from 4 to 6 (#48833)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4 to 6. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/actions/upload-artifact/releases">actions/upload-artifact's releases</a>.</em></p> <blockquote> <h2>v6.0.0</h2> <h2>v6 - What's new</h2> <blockquote> <p>[!IMPORTANT] actions/upload-artifact@ v6 now runs on Node.js 24 (<code>runs.using: node24</code>) and requires a minimum Actions Runner version of 2.327.1. If you are using self-hosted runners, ensure they are updated before upgrading.</p> </blockquote> <h3>Node.js 24</h3> <p>This release updates the runtime to Node.js 24. v5 had preliminary support for Node.js 24, however this action was by default still running on Node.js 20. Now this action by default will run on Node.js 24.</p> <h2>What's Changed</h2> <ul> <li>Upload Artifact Node 24 support by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/upload-artifact/pull/719">actions/upload-artifact#719</a></li> <li>fix: update <code>@actions/artifact</code> for Node.js 24 punycode deprecation by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/upload-artifact/pull/744">actions/upload-artifact#744</a></li> <li>prepare release v6.0.0 for Node.js 24 support by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/upload-artifact/pull/745">actions/upload-artifact#745</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/upload-artifact/compare/v5.0.0...v6.0.0">https://github.com/actions/upload-artifact/compare/v5.0.0...v6.0.0</a></p> <h2>v5.0.0</h2> <h2>What's Changed</h2> <p><strong>BREAKING CHANGE:</strong> this update supports Node <code>v24.x</code>. This is not a breaking change per-se but we're treating it as such.</p> <ul> <li>Update README.md by <a href="https://github.com/GhadimiR"><code>@GhadimiR</code></a> in <a href="https://redirect.github.com/actions/upload-artifact/pull/681">actions/upload-artifact#681</a></li> <li>Update README.md by <a href="https://github.com/nebuk89"><code>@nebuk89</code></a> in <a href="https://redirect.github.com/actions/upload-artifact/pull/712">actions/upload-artifact#712</a></li> <li>Readme: spell out the first use of GHES by <a href="https://github.com/danwkennedy"><code>@danwkennedy</code></a> in <a href="https://redirect.github.com/actions/upload-artifact/pull/727">actions/upload-artifact#727</a></li> <li>Update GHES guidance to include reference to Node 20 version by <a href="https://github.com/patrikpolyak"><code>@patrikpolyak</code></a> in <a href="https://redirect.github.com/actions/upload-artifact/pull/725">actions/upload-artifact#725</a></li> <li>Bump <code>@ actions/artifact</code> to <code>v4.0.0</code></li> <li>Prepare <code>v5.0.0</code> by <a href="https://github.com/danwkennedy"><code>@danwkennedy</code></a> in <a href="https://redirect.github.com/actions/upload-artifact/pull/734">actions/upload-artifact#734</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/GhadimiR"><code>@GhadimiR</code></a> made their first contribution in <a href="https://redirect.github.com/actions/upload-artifact/pull/681">actions/upload-artifact#681</a></li> <li><a href="https://github.com/nebuk89"><code>@nebuk89</code></a> made their first contribution in <a href="https://redirect.github.com/actions/upload-artifact/pull/712">actions/upload-artifact#712</a></li> <li><a href="https://github.com/danwkennedy"><code>@danwkennedy</code></a> made their first contribution in <a href="https://redirect.github.com/actions/upload-artifact/pull/727">actions/upload-artifact#727</a></li> <li><a href="https://github.com/patrikpolyak"><code>@patrikpolyak</code></a> made their first contribution in <a href="https://redirect.github.com/actions/upload-artifact/pull/725">actions/upload-artifact#725</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/upload-artifact/compare/v4...v5.0.0">https://github.com/actions/upload-artifact/compare/v4...v5.0.0</a></p> <h2>v4.6.2</h2> <h2>What's Changed</h2> <ul> <li>Update to use artifact 2.3.2 package & prepare for new upload-artifact release by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/upload-artifact/pull/685">actions/upload-artifact#685</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> made their first contribution in <a href="https://redirect.github.com/actions/upload-artifact/pull/685">actions/upload-artifact#685</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/upload-artifact/compare/v4...v4.6.2">https://github.com/actions/upload-artifact/compare/v4...v4.6.2</a></p> <h2>v4.6.1</h2> <h2>What's Changed</h2> <ul> <li>Update to use artifact 2.2.2 package by <a href="https://github.com/yacaovsnc"><code>@yacaovsnc</code></a> in <a href="https://redirect.github.com/actions/upload-artifact/pull/673">actions/upload-artifact#673</a></li> </ul> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
Raúl Cumplido
|
985b16ec27 |
GH-48582: [CI][GPU][C++][Python] Add new CUDA jobs using the new self-hosted runners (#48583)
### Rationale for this change The CUDA jobs stopped working when Voltron Data infrastructure went down. We have set up with ASF Infra a [runs-on](https://runs-on.com/runners/gpu/) solution to run CUDA runners. ### What changes are included in this PR? Add the new workflow for `cuda_extra.yml` with CI jobs that use the runs-on CUDA runners. Due to the underlying instances having CUDA 12.9 the jobs to be run are: - AMD64 Ubuntu 22 CUDA 11.7.1 - AMD64 Ubuntu 24 CUDA 12.9.0 - AMD64 Ubuntu 22 CUDA 11.7.1 Python - AMD64 Ubuntu 24 CUDA 12.9.0 Python A follow up issue has been created to add jobs for CUDA 13, see: https://github.com/apache/arrow/issues/48783 A new label `CI: Extra: CUDA` has also been created. ### Are these changes tested? Yes via CI ### Are there any user-facing changes? No * GitHub Issue: #48582 Authored-by: Raúl Cumplido <raulcumplido@gmail.com> Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com> |
||
|
Raúl Cumplido
|
2805afd27e |
GH-48782: [Docs][CI] Skip Markdown files with doxygen and trigger Docs job on PR when files are modified (#48786)
### Rationale for this change Docs jobs are currently failing with Doxygen failure due to parsing the `cpp/src/arrow/flight/sql/odbc/README.md` file. ### What changes are included in this PR? Minor fix to escape path and add Skip markdown files when running doxygen on doxygen config. Add trigger for Complete docs workflow on Pull requests when documentation is modified. ### Are these changes tested? Yes locally. ### Are there any user-facing changes? No * GitHub Issue: #48782 Authored-by: Raúl Cumplido <raulcumplido@gmail.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
Hyukjin Kwon
|
fe5e0e5eec |
GH-48780: [CI] Add missing permissions for reusable workflow calls (#48778)
### Rationale for this change Workflows `cpp_extra.yml`, `r_extra.yml` and `package_linux.yml` call reusable workflows (`check_labels.yml` and `report_ci.yml`) that require specific permissions. When #48771 added explicit permissions to these reusable workflows, the calling workflows were not updated to give those permissions. This caused `startup_failure` errors when these workflows were triggered on pull requests. Here are example failures: https://github.com/apache/arrow/actions/runs/20809257825 and https://github.com/apache/arrow/actions/runs/20803198596 ### What changes are included in this PR? Added missing permissions to the workflows ### Are these changes tested? Yes, I tested them within this PR. ### Are there any user-facing changes? No, dev-only. * GitHub Issue: #48780 Authored-by: Hyukjin Kwon <gurwls223@apache.org> Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com> |
||
|
Raúl Cumplido
|
076052300b |
GH-48461: [R][CI] Migrate Azure pipelines to GitHub actions (#48585)
### Rationale for this change We have been consolidating our CI to use GitHub actions during the last years. There are only a handful of CI jobs on Azure pipelines and all of them are R related. From a maintenance perspective using a single UI or having to update less secrets when we need to update them is easier. ### What changes are included in this PR? Remove all the jobs that where using Azure pipelines from our crossbow `tasks.yml` workflows. Migrate them to a new `r_extra.yml` workflow that will be executed following the same pattern as CPP Extra or Linux Packages. Some cleanup for unnecessary Azure macros now that there are no Azure pipelines on crossbow tasks. ### Are these changes tested? Yes via CI. ### Are there any user-facing changes? No * GitHub Issue: #48461 Authored-by: Raúl Cumplido <raulcumplido@gmail.com> Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com> |
||
|
Hyukjin Kwon
|
fdeac0b983 |
GH-48770: [CI] Add missing permissions declaration to workflows (#48771)
### Rationale for this change
Adds missing `permissions:` declaration to workflows, following the security best practices (see also #35708).
### What changes are included in this PR?
Adds workflow-level permissions to workflows
### Are these changes tested?
Tested in
|
||
|
Alina (Xi) Li
|
45cf928756 |
GH-47787: [C++][FlightRPC] ODBC msi Windows installer (#48054)
### Rationale for this change Implement ODBC installer code for Windows. The primary platform for ODBC is MSVC Windows. The version is set to 1.0.0. ### What changes are included in this PR? - ODBC msi installer - Added components to other parts of Arrow, so we can select ODBC component for cpack ### Are these changes tested? Tested locally on MSVC Windows ### Are there any user-facing changes? N/A * GitHub Issue: #47787 Authored-by: Alina (Xi) Li <alina.li@improving.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
|
Hyukjin Kwon
|
28855e0be2 |
GH-48757: [CI] Update arrow/.github /CODEOWNERS (#48758)
### Rationale for this change The CODEOWNERS file contained references to CI configuration files that no longer exist in the repository. Travis CI and AppVeyor have been replaced by GitHub Actions by |
||
|
Sutou Kouhei
|
13a607a898 |
GH-48606: [CI][GLib] Increase NuGet timeout for vcpkg cache (#48638)
### Rationale for this change There are timeout errors for NuGet push to cache vcpkg packages. ### What changes are included in this PR? Increase push timeout to 600s from 300s. See also: https://learn.microsoft.com/en-us/vcpkg/reference/binarycaching ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: #48606 Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com> |
||
|
dependabot[bot]
|
bd8ab75009 |
MINOR: [CI] Bump actions/cache from 4 to 5 (#48620)
Bumps [actions/cache](https://github.com/actions/cache) from 4 to 5. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/actions/cache/releases">actions/cache's releases</a>.</em></p> <blockquote> <h2>v5.0.0</h2> <blockquote> <p>[!IMPORTANT] <strong><code>actions/cache@ v5</code> runs on the Node.js 24 runtime and requires a minimum Actions Runner version of <code>2.327.1</code>.</strong></p> <p>If you are using self-hosted runners, ensure they are updated before upgrading.</p> </blockquote> <hr /> <h2>What's Changed</h2> <ul> <li>Upgrade to use node24 by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/cache/pull/1630">actions/cache#1630</a></li> <li>Prepare v5.0.0 release by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/cache/pull/1684">actions/cache#1684</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/cache/compare/v4.3.0...v5.0.0">https://github.com/actions/cache/compare/v4.3.0...v5.0.0</a></p> <h2>v4.3.0</h2> <h2>What's Changed</h2> <ul> <li>Add note on runner versions by <a href="https://github.com/GhadimiR"><code>@GhadimiR</code></a> in <a href="https://redirect.github.com/actions/cache/pull/1642">actions/cache#1642</a></li> <li>Prepare <code>v4.3.0</code> release by <a href="https://github.com/Link"><code>@Link</code></a>- in <a href="https://redirect.github.com/actions/cache/pull/1655">actions/cache#1655</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/GhadimiR"><code>@GhadimiR</code></a> made their first contribution in <a href="https://redirect.github.com/actions/cache/pull/1642">actions/cache#1642</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/cache/compare/v4...v4.3.0">https://github.com/actions/cache/compare/v4...v4.3.0</a></p> <h2>v4.2.4</h2> <h2>What's Changed</h2> <ul> <li>Update README.md by <a href="https://github.com/nebuk89"><code>@nebuk89</code></a> in <a href="https://redirect.github.com/actions/cache/pull/1620">actions/cache#1620</a></li> <li>Upgrade <code>@ actions/cache</code> to <code>4.0.5</code> and move <code>@ protobuf-ts/plugin</code> to dev depdencies by <a href="https://github.com/Link"><code>@Link</code></a>- in <a href="https://redirect.github.com/actions/cache/pull/1634">actions/cache#1634</a></li> <li>Prepare release <code>4.2.4</code> by <a href="https://github.com/Link"><code>@Link</code></a>- in <a href="https://redirect.github.com/actions/cache/pull/1636">actions/cache#1636</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/nebuk89"><code>@nebuk89</code></a> made their first contribution in <a href="https://redirect.github.com/actions/cache/pull/1620">actions/cache#1620</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/cache/compare/v4...v4.2.4">https://github.com/actions/cache/compare/v4...v4.2.4</a></p> <h2>v4.2.3</h2> <h2>What's Changed</h2> <ul> <li>Update to use <code>@actions/cache</code> 4.0.3 package & prepare for new release by <a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> in <a href="https://redirect.github.com/actions/cache/pull/1577">actions/cache#1577</a> (SAS tokens for cache entries are now masked in debug logs)</li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/salmanmkc"><code>@salmanmkc</code></a> made their first contribution in <a href="https://redirect.github.com/actions/cache/pull/1577">actions/cache#1577</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/cache/compare/v4.2.2...v4.2.3">https://github.com/actions/cache/compare/v4.2.2...v4.2.3</a></p> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/actions/cache/blob/main/RELEASES.md">actions/cache's changelog</a>.</em></p> <blockquote> <h1>Releases</h1> <h2>Changelog</h2> <h3>5.0.1</h3> <ul> <li>Update <code>@ azure/storage-blob</code> to <code>^12.29.1</code> via <code>@ actions/cache@ 5.0.1</code> <a href="https://redirect.github.com/actions/cache/pull/1685">#1685</a></li> </ul> <h3>5.0.0</h3> <blockquote> <p>[!IMPORTANT] <code>actions/cache@ v5</code> runs on the Node.js 24 runtime and requires a minimum Actions Runner version of <code>2.327.1</code>. If you are using self-hosted runners, ensure they are updated before upgrading.</p> </blockquote> <h3>4.3.0</h3> <ul> <li>Bump <code>@ actions/cache</code> to <a href="https://redirect.github.com/actions/toolkit/pull/2132">v4.1.0</a></li> </ul> <h3>4.2.4</h3> <ul> <li>Bump <code>@ actions/cache</code> to v4.0.5</li> </ul> <h3>4.2.3</h3> <ul> <li>Bump <code>@ actions/cache</code> to v4.0.3 (obfuscates SAS token in debug logs for cache entries)</li> </ul> <h3>4.2.2</h3> <ul> <li>Bump <code>@ actions/cache</code> to v4.0.2</li> </ul> <h3>4.2.1</h3> <ul> <li>Bump <code>@ actions/cache</code> to v4.0.1</li> </ul> <h3>4.2.0</h3> <p>TLDR; The cache backend service has been rewritten from the ground up for improved performance and reliability. <a href="https://github.com/actions/cache">actions/cache</a> now integrates with the new cache service (v2) APIs.</p> <p>The new service will gradually roll out as of <strong>February 1st, 2025</strong>. The legacy service will also be sunset on the same date. Changes in these release are <strong>fully backward compatible</strong>.</p> <p><strong>We are deprecating some versions of this action</strong>. We recommend upgrading to version <code>v4</code> or <code>v3</code> as soon as possible before <strong>February 1st, 2025.</strong> (Upgrade instructions below).</p> <p>If you are using pinned SHAs, please use the SHAs of versions <code>v4.2.0</code> or <code>v3.4.0</code></p> <p>If you do not upgrade, all workflow runs using any of the deprecated <a href="https://github.com/actions/cache">actions/cache</a> will fail.</p> <p>Upgrading to the recommended versions will not break your workflows.</p> <h3>4.1.2</h3> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
Antoine Pitrou
|
30809c6f48 |
GH-45885: [C++] Require C++20 (#48414)
### Rationale for this change We decided to migrate Arrow C++ to C++20 in this discussion: https://lists.apache.org/thread/48zlj0dn2y0f53y2k37qsr90y781wfnj ### What changes are included in this PR? 1. Build configuration updates (CMake files etc.) to build with C++20 instead of C++17 2. C++-level fixes to ensure compilation succeeds: * Workarounds for deprecation of atomic access on `std::shared_ptr<T>` (the replacement `std::atomic<std::shared_ptr<T>>` is unfortunately not supported in all standard library implementations) * Workaround for [Abseil ABI issues](https://github.com/abseil/abseil-cpp/issues/1624) * Trivial update to `arrow/util/string.h` to call a C++20 API (to validate that C++20 is actually enabled) 3. CI configuration updates to get enough C++20 support on the various platforms: * Bump default clang and LLVM versions to 18 * Use AlmaLinux 10 for release verification (AlmaLinux 8 is still tested on other CI builds) ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes, Arrow C++ will now require a C++20-compliant compiler. * GitHub Issue: #45885 Authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Antoine Pitrou <antoine@python.org> |
||
|
Nic Crane
|
c988d17c9c |
GH-45449: [R][CI] Remove OpenSSL 1.x builds (#48297)
### Rationale for this change We support old OpenSSL versions which are now deprecated. It complicates our CI by having unnecessary things running too. ### What changes are included in this PR? Stop supporting OpenSSL < 3.0 ### Are these changes tested? Will trigger CI ### Are there any user-facing changes? Not unless they're using deprecated versions of OpenSSL, which isn't something we should support anyway * GitHub Issue: #45449 Authored-by: Nic Crane <thisisnic@gmail.com> Signed-off-by: Nic Crane <thisisnic@gmail.com> |
||
|
Sutou Kouhei
|
2e12201dc3 |
GH-40735: [Packaging][CentOS] Drop support for CentOS 7 (#48550)
### Rationale for this change There is no sponsor to maintain EOL-ed CentOS 7. ### What changes are included in this PR? Remove CentOS 7 packages related configurations. ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes. * GitHub Issue: #40735 Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com> |