Morph MORPH
SIGN IN SIGN UP
apache / arrow UNCLAIMED

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics

0 0 2 C++
Normal View History Raw
ARROW-1662: Move to using Homebrew/bundle and Brewfile improves osx dependency management Author: Stephen G <stephen@egroat.com> Author: Stephen <stephengroat@users.noreply.github.com> Author: Stephen Groat <stephengroat@Stephens-MacBook-Pro.local> Closes #1143 from stephengroat/osx-brew and squashes the following commits: 81d4ac15 [Stephen G] Merge branch 'osx-brew' of https://github.com/stephengroat/arrow into osx-brew 62a79291 [Stephen G] Add license headers cee779b1 [Stephen] increase to max travis_wait 583ec0a7 [Stephen G] Move to individual brewfiles 65148557 [Stephen] add travis wait to get around timeouts 6faef0ce [Stephen Groat] Move to using Homebrew/bundle and Brewfile
2017-10-10 23:47:29 -04:00
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
ARROW-14684: [CI][C++] Use aws-sdk-cpp package on macOS Closes #11685 from kou/ci-cpp-macos-aws-sdk-cpp Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
2021-11-16 05:45:16 +09:00
brew "aws-sdk-cpp"
ARROW-16801: [CI][C++] Use the specified MinIO instead of MinIO from Homewbrew (#13362) Our tests have problem with recent MinIO. We pin the version of MinIO to make our test stable. See also: ARROW-16722/#13299 Lead-authored-by: Raúl Cumplido <raulcumplido@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Co-authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
2022-06-16 05:00:47 +02:00
brew "bash"
ARROW-1662: Move to using Homebrew/bundle and Brewfile improves osx dependency management Author: Stephen G <stephen@egroat.com> Author: Stephen <stephengroat@users.noreply.github.com> Author: Stephen Groat <stephengroat@Stephens-MacBook-Pro.local> Closes #1143 from stephengroat/osx-brew and squashes the following commits: 81d4ac15 [Stephen G] Merge branch 'osx-brew' of https://github.com/stephengroat/arrow into osx-brew 62a79291 [Stephen G] Add license headers cee779b1 [Stephen] increase to max travis_wait 583ec0a7 [Stephen G] Move to individual brewfiles 65148557 [Stephen] add travis wait to get around timeouts 6faef0ce [Stephen Groat] Move to using Homebrew/bundle and Brewfile
2017-10-10 23:47:29 -04:00
brew "boost"
ARROW-5660: [GLib][CI] Use Xcode 10.2 Author: Sutou Kouhei <kou@clear-code.com> Closes #4641 from kou/glib-ci-macos-10.2 and squashes the following commits: e7a926823 <Sutou Kouhei> Use Xcode 10.2
2019-06-20 20:30:26 -07:00
brew "brotli"
GH-37923: [R] Move macOS build system to nixlibs.R (#37684) This PR modifies the build system of the R package to no longer rely on auto/homebrew. Instead this PR adds the infrastructure and code paths to use the same type of pre-compiled libarrow binaries as we use for Linux. The main difference is the use of the binaries even on CRAN (as we previously also used binaries in form of brew bottles). The addition of the new artifacts to tasks.yml should ensure that they get uploaded to the nightly repo as well as to the artifactory during the release (@ kou please confirm). A summary of the changes in this PR: - update `r/configure` and `r/tools/nixlibs.R` to enable the source build on macOS and usage of precompiled binaries using the existing mechanism to test compile a program to detect the exisitng openssl version - added tests for the changes in nixlibs.R - update the binary allow-list - Add the build jobs for libarrow binaries for arm64 and x86_64 macos with openssl 1.1 and 3.0 to the `r-binary-packages` job - Use the binaries to build the nightly packages - bump snappy version to 1.1.10 (and patch it on 10.13) due to build issues with the current version. This also touches on a number of issues in regards to a sanitizer issue we have had for a long time: #32562 #31766 - Disable the centos binary test step: #37922 Follow up issues: - #37921 - #37941 - #37945 * Closes: #37923 Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Co-authored-by: Jonathan Keane <jkeane@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
2023-10-05 14:48:44 +02:00
brew "bzip2"
ARROW-7101: [CI] Refactor docker-compose setup and use it with GitHub Actions ## Projecting ideas from ursabot ### Parametric docker images The images are better parameterized now, meaning that we can build more variant of the same service. Couple of examples: ```console UBUNTU=16.04 docker-compose build ubuntu-cpp ARCH=arm64v8 UBUNTU=18.04 docker-compose build ubuntu-cpp PYTHON=3.6 docker-compose build conda-python ARCH=arm32v7 PYTHON=3.6 PANDAS=0.25 docker-compose build conda-python-pandas ``` Each variant has it's own docker image following a string naming schema: `{org}/{arch}-{platform}-{platform-version}[[-{variant}-{variant-version}]..]:latest` ### Use *_build.sh and *_test.sh for each job The docker images provide the environment, and each language backend usually should implement two scripts, a `build.sh` and a `test.sh`. This way dependent build like the docker python, r or c glib are able to reuse the build script of the ancestor without running its tests. With small enough scripts, if the environment is properly set up even the non-docker builds should be reproducible locally. GitHub Actions support bash scripts across all three platforms, so we can reuse the same `*_build.sh` and `*_test.sh` scripts to execute the builds either in docker, on the CI or locally. ## Using GitHub Actions for running the builds Regardless of the CI we're going to choose, the isolation constraint of different platforms requires some sort of virtualisation. Currently linux (and windows, but I have not tried it yet) has lightweight containerisation, so we should keep the linux builds isolated in docker containers. The rest of the platforms (windows and macOS) should be executed on the CI system. GitHub Actions support all three major platforms, linux, windows and macOS. I've added cross platform builds for a couple of languages, like Rust, and Go, the rest are work in progress. ### Workflow A workflow should define all builds of a language, mostly because the path filters can be defined on workflow level. For example the python builds should be triggered if either a cpp/** or a python/** file changes which can be covered in the same workflow file. ## Feature parity with the current builds Reaching feature parity with all of the builds below is not a goal for this PR, the difficult ones should at least have a tracking JIRA ticket. ### Travis-CI - [x] **Lint, Release tests**: - `Lint / C++, Python, R, Rust, Docker, RAT` - `Dev / Source Release` - [x] **C++ unit tests w/ conda-forge toolchain, coverage**: without coverage - `C++ / AMD64 Conda C++` - [x] **Python 3.6 unit tests, conda-forge toolchain, coverage**: without coverage - `Python / AMD64 Conda Python 3.6` - [x] **[OS X] C++ w/ Xcode 9.3**: - `C++ / AMD64 MacOS 10.14 C++`: with Xcode 10.3 - [x] **[OS X] Python w/ Xcode 9.3**: - `Python / AMD64 MacOS 10.14 Python 3`: with Xcode 10.3 - [x] **Java OpenJDK8 and OpenJDK11**: - `Java / AMD64 Debian Java JDK 8 Maven 3.5.2` - `Java / AMD64 Debian Java JDK 11 Maven 3.6.2` - [x] **Protocol / Flight Integration Tests**: - `Dev / Protocol Test` - [x] **NodeJS**: without running lint and coverage - `NodeJS / AMD64 Debian NodeJS 11` - [x] **C++ & GLib & Ruby w/ gcc 5.4**: - `C++ / AMD64 Debian 10 C++`: with GCC 8.3 - `C++ / AMD64 Ubuntu 16.04 C++`: with GCC 5.4 - `C++ / AMD64 Ubuntu 18.04 C++`: with GCC 7.4 - `C GLib / AMD64 Ubuntu 18.04 C GLib` - `Ruby / AMD64 Ubuntu 18.04 Ruby` - [x] **[OS X] C++ & GLib & Ruby w/ XCode 10.2 & Homebrew** - `C++ / AMD64 MacOS 10.14 C++`: with Xcode 10.3 - `C GLib / AMD64 MacOS 10.14 C Glib`: with Xcode 10.3 - `Ruby / AMD64 MacOS 10.14 Ruby`: with Xcode 10.3 - [x] **Go**: without coverage - `Go / AMD64 Debian Go 1.12` - [x] **R (with and without libarrow)**: - `R / AMD64 Conda R 3.6`: with libarrow - `R / AMD64 Ubuntu 18.04 R 3.6` with libarrow ### Appveyor - ~JOB=Build, GENERATOR=Ninja, CONFIGURATION=Release, APPVEYOR_BUILD_WORKER_IMAGE=Visual Studio 2017~ - ~JOB=Toolchain, GENERATOR=Ninja, CONFIGURATION=Release, ARROW_S3=ON, ARROW_BUILD_FLIGHT=ON, ARROW_BUILD_GANDIVA=ON~ - ~JOB=Build_Debug, GENERATOR=Ninja, CONFIGURATION=Debug~ - ~JOB=MinGW32, MINGW_ARCH=i686, MINGW_PACKAGE_PREFIX=mingw-w64-i686, MINGW_PREFIX=c:\msys64\mingw32, MSYSTEM=MINGW32, USE_CLCACHE=false~ - ~JOB=MinGW64, MINGW_ARCH=x86_64, MINGW_PACKAGE_PREFIX=mingw-w64-x86_64, MINGW_PREFIX=c:\msys64\mingw64, MSYSTEM=MINGW64, USE_CLCACHE=false~ - [x] **JOB=Rust, TARGET=x86_64-pc-windows-msvc, USE_CLCACHE=false**: - `Rust / AMD64 Windows 2019 Rust nightly-2019-09-25` - [x] **JOB=C#, APPVEYOR_BUILD_WORKER_IMAGE=Visual Studio 2017, USE_CLCACHE=false** - `C# / AMD64 Windows 2019 C# 2.2.103` - [x] **JOB=Go, MINGW_PACKAGE_PREFIX=mingw-w64-x86_64 ...**: - `Go / AMD64 Windows 2019 Go 1.12` - ~JOB=R with libarrow, USE_CLCACHE=false, TEST_R_WITH_ARROW=TRUE, RWINLIB_LOCAL=%APPVEYOR_BUILD_FOLDER%\libarrow.zip~ ### Github Actions - [x] **Windows MSVC C++ / Build (Visual Studio 16 2019)**: - `C++ / AMD64 Windows 2019 C++`: without tests - [x] **Windows MSVC C++ / Build (Visual Studio 15 2017)**: - `C++ / AMD64 Windows 2016 C++`: without tests - [x] **Linux docker-compose / Test (C++ w/ clang-7 & system packages)**: all have llvm for gandiva but the compiler is set to gcc - `C++ / AMD64 Debian 10 C++`: with GCC 8.3 - `C++ / AMD64 Ubuntu 16.04 C++`: with GCC 5.4 - `C++ / AMD64 Ubuntu 18.04 C++`: with GCC 7.4 - [x] **Linux docker-compose / Test (Rust)**: without rustfmt - `Rust / AMD64 Debian Rust nightly-2019-09-25` - [x] **Linux docker-compose / Test (Lint, Release tests)**: - `Lint / C++, Python, R, Rust, Docker, RAT` - `Dev / Source Release` ### Nightly Crossbow tests The packaging builds are out of the scope if this PR, but the nightly **dockerized test** task are in. Nightly tests: - [x] docker-r - [x] docker-r-conda - [x] docker-r-sanitizer - [x] docker-rust - [x] docker-cpp - [x] docker-cpp-cmake32 - [x] docker-cpp-release - [x] docker-cpp-static-only - [x] docker-c_glib - [x] docker-go - [x] docker-python-2.7 - [x] docker-python-3.6 - [x] docker-python-3.7 - [x] docker-python-2.7-nopandas - [x] docker-python-3.6-nopandas - [x] docker-java - [x] docker-js - [x] docker-docs - [x] docker-lint - [x] docker-iwyu: included in the lint - [x] docker-clang-format: included in the lint - [x] docker-pandas-master - [x] docker-dask-integration - [x] docker-hdfs-integration - [x] docker-spark-integration - [x] docker-turbodbc-integration # TODOs left: - [x] Fix the Apidoc generation for c_glib - [x] Fix the JNI test for Gandiva and ORC - [x] Test that crossbow tests are passing - ~Optionally restore the travis configuration to incrementally decommission old builds~ ## Follow-up JIRAs: - [Archery] Consider porting the docker tool of ursabot to archery - [Archery] Consider to use archery with or instead of the pre-commit hooks - [Archery] Create a wrapper script in archery for docker compose in order to run the containers with the host's user and group - [C++] GCC 5.4.0 has a compile errors, reproduce with UBUNTU=16.04 docker-compose run ubuntu-cpp - [C++][CI] Test the ported fuzzit integration image - [C++][CI] Turn off unnecessary features in the integration tests (spark/turbodbc/dask/hdfs) - [C++][CI] Revisit ASAN UBSAN settings in every C++ based image - [CI] Consider re-adding the removed debian testing image is removed - [Go][CI] Pre-install the go dependencies in the dockerfile using go get - [JS][CI] Pre-install the JS dependencies in the dockerfile - [Rust][CI] Pre-install the rust dependencies in the dockerfile - [Java][CI] Pre-install the java dependencies in the dockerfile - [Ruby][CI] Pre-install the ruby dependencies in the dockerfile and remove it from the test script - [C#][CI] Pre-install the C# dependencies in the dockerfile - [R][CI] Fix the r-sanitizer build https://issues.apache.org/jira/browse/ARROW-6957 - [GLIB][MacOS] Fail to execute lua examples (fails to load 'lgi.corelgilua51' despite that lgi is installed) - [C++][CMake] Automatically set ARROW_GANDIVA_PC_CXX_FLAGS for conda and OSX sdk (see cpp_build.sh) - [C++][CI] Hiveserver2 instegration test fails to connect to impala container - [CI][Spark] Support specific Spark version in the integration tet including latest - [JS][CI] Move nodejs linting from js_build.sh to archery - [Python][CI] create a docker image for python ASV benchmarks and fix the script - [CI] Find a short but related prefix for the env vars used for the docker-compose file to prevent collisions - [C#] the docker container fails to run because of the ubuntu host versions, see https://github.com/dotnet/core/issues/3509 - [C++][Windows] Enable more features on the windows GHA build - [Doc] document docker-compose usage in the developer sphinx guide - [CI][C++] Add .ccache to the docker-compose mounts - [Archery][CI] Refactor the ci/scripts to a sourceable bash functions or to archery directly - [C++][CI] Use scripts/util_coredump.sh to show automatic backtraces - [C++] Fix the hanging C++ tests in Windows 2019 - [CI] Ask INFRA to set up the DOCKERHUB_* secrets for GitHub actions - [C++][CI] Running Gandiva tests fails on Fedora: Reproduce with: `docker-compose run -e ARROW_GANDIVA=ON fedora-cpp` ``` Running gandiva-internals-test, redirecting output into /build/cpp/build/test-logs/gandiva-internals-test.txt (attempt 1/1) 1364 : CommandLine Error: Option 'x86-experimental-vector-widening-legalization' registered more than once! 1365 LLVM ERROR: inconsistency in registered CommandLine options 1366 /build/cpp/src/gandiva ``` - [JS][CI] NodeJS build fails on Github Actions Windows node ``` > NODE_NO_WARNINGS=1 gulp build # 'NODE_NO_WARNINGS' is not recognized as an internal or external command, # operable program or batch file. # npm ERR! code ELIFECYCLE # npm ERR! errno 1 # npm ERR! apache-arrow@1.0.0-SNAPSHOT build: `NODE_NO_WARNINGS=1 gulp build` # npm ERR! Exit status 1 # npm ERR! # npm ERR! Failed at the apache-arrow@1.0.0-SNAPSHOT build script. # npm ERR! This is probably not a problem with npm. There is likely additional logging output above. ``` Closes #5589 from kszucs/docker-refactor and squashes the following commits: 5105d12e6 <Krisztián Szűcs> Rename pull-request folder to dev_cron e9e9a7eec <Krisztián Szűcs> Use underscores for naming the workflow files a92c99d03 <Krisztián Szűcs> Disable hanging C++ tests on windows f158c89b5 <Krisztián Szűcs> Attempt to push from apache/arrow master; Don't push from crossbow tasks 0e1d470a1 <Krisztián Szűcs> Turn off ORC on macOS C++ test due to link error 258db5cff <Krisztián Szűcs> Only push docker images from apache/arrow repository acdfcf086 <Krisztián Szűcs> Remove ORC from the brewfile 5102b85b1 <Krisztián Szűcs> Fix nodeJS workflow 032d6a388 <Krisztián Szűcs> Turn off 2 python builds 7f15b97a8 <Krisztián Szűcs> Filter branches 48b8d128a <Krisztián Szűcs> Fix workflows 36ad9d297 <Krisztián Szűcs> Disable builds 0f603af0c <Krisztián Szűcs> master only and cron workflows 28cc2d78d <Krisztián Szűcs> Rename Java JNI workflow bcd8af7b7 <Krisztián Szűcs> Port the remaining travis utility scripts ed5688154 <Krisztián Szűcs> Usage comments; recommend installing pandas from the docs because of its removal from conda_env_python 3c8c023ce <Krisztián Szűcs> Use Arch in volumes; some comments; remove conda version 'latest' from the images 771b023a8 <Krisztián Szűcs> Cleanup files; separate JNI builds 97ff8a122 <Krisztián Szűcs> Push docker images only from master dc00b4297 <Krisztián Szűcs> Enable path filters e0e2e1f46 <Krisztián Szűcs> Fix pandas master build 3814e0828 <Krisztián Szűcs> Fix manylinux volumes c18edda70 <Krisztián Szűcs> Add CentOS version to the manylinux image names c8b9dd6b1 <Krisztián Szűcs> Missing --pyargs argument for the python test command 33e646981 <Krisztián Szűcs> Turn off gandiva and flight for the HDFS test b9c547889 <Krisztián Szűcs> Refactor docker-compose file and use it with github actions. Authored-by: Krisztián Szűcs <szucs.krisztian@gmail.com> Signed-off-by: Krisztián Szűcs <szucs.krisztian@gmail.com>
2019-11-12 11:07:48 +01:00
brew "c-ares"
GH-37923: [R] Move macOS build system to nixlibs.R (#37684) This PR modifies the build system of the R package to no longer rely on auto/homebrew. Instead this PR adds the infrastructure and code paths to use the same type of pre-compiled libarrow binaries as we use for Linux. The main difference is the use of the binaries even on CRAN (as we previously also used binaries in form of brew bottles). The addition of the new artifacts to tasks.yml should ensure that they get uploaded to the nightly repo as well as to the artifactory during the release (@ kou please confirm). A summary of the changes in this PR: - update `r/configure` and `r/tools/nixlibs.R` to enable the source build on macOS and usage of precompiled binaries using the existing mechanism to test compile a program to detect the exisitng openssl version - added tests for the changes in nixlibs.R - update the binary allow-list - Add the build jobs for libarrow binaries for arm64 and x86_64 macos with openssl 1.1 and 3.0 to the `r-binary-packages` job - Use the binaries to build the nightly packages - bump snappy version to 1.1.10 (and patch it on 10.13) due to build issues with the current version. This also touches on a number of issues in regards to a sanitizer issue we have had for a long time: #32562 #31766 - Disable the centos binary test step: #37922 Follow up issues: - #37921 - #37941 - #37945 * Closes: #37923 Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Co-authored-by: Jonathan Keane <jkeane@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
2023-10-05 14:48:44 +02:00
brew "curl"
ARROW-5222: [Python] Revise pyarrow installation instructions for macOS This patch adds some docs and some links to previously existing documentation to help clarify the problems outlined on https://issues.apache.org/jira/browse/ARROW-5222. It also resolves the multiple-Brewfile issue by factoring out a common cpp/Brewfile. There are also some assorted formatting cleanups here too. Author: Neal Richardson <neal.p.richardson@gmail.com> Closes #4231 from nealrichardson/pyarrow-macos-dev-instructions and squashes the following commits: 716f2607f <Neal Richardson> Brewfile tweak 6014fac37 <Neal Richardson> Tweak language about conda on macos 13385853e <Neal Richardson> Update .gitignore 33a041c7f <Neal Richardson> Factor out a cpp/Brewfile and document it (thanks @kou) 9d333e517 <Neal Richardson> Use :ref: to link to other sections 4158d70be <Neal Richardson> Revise pyarrow installation instructions for macOS
2019-05-09 18:44:49 +02:00
brew "ccache"
brew "cmake"
brew "git"
brew "glog"
ARROW-17081: [Java][Datasets] Move JNI build configuration from cpp/ to java/ (#13911) Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
2022-09-06 14:52:52 +09:00
brew "googletest"
GH-35987: [C++] Unpin brew protobuf version (#36087) Unpin brew protobuf version because of an upstream patch. * Closes: #35987 Authored-by: Jin Shang <shangjin1997@gmail.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
2023-06-16 14:55:02 +08:00
brew "grpc"
GH-48575: [C++][FlightRPC] Standalone ODBC macOS CI (#48577) ### Rationale for this change #48575 ### What changes are included in this PR? - Add new ODBC workflow for macOS Intel 15 and 14 arm64. - Added ODBC build fixes to enable build on macOS CI. ### Are these changes tested? Tested in CI and local macOS Intel and M1 environments. ### Are there any user-facing changes? N/A * GitHub Issue: #48575 Lead-authored-by: Alina (Xi) Li <alina.li@improving.com> Co-authored-by: justing-bq <62349012+justing-bq@users.noreply.github.com> Co-authored-by: Victor Tsang <victor.tsang@improving.com> Co-authored-by: Alina (Xi) Li <alinal@bitquilltech.com> Co-authored-by: vic-tsang <victor.tsang@improving.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
2026-02-07 03:54:25 -08:00
brew "libiodbc"
GH-45784: [C++] Unpin LLVM and OpenSSL in Brewfile (#45785) ### Rationale for this change We don't need to pin LLVM and OpenSSL now. ### What changes are included in this PR? Unpin LLVM and OpenSSL. ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: #45784 Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
2025-03-18 15:55:32 +09:00
brew "llvm"
ARROW-5660: [GLib][CI] Use Xcode 10.2 Author: Sutou Kouhei <kou@clear-code.com> Closes #4641 from kou/glib-ci-macos-10.2 and squashes the following commits: e7a926823 <Sutou Kouhei> Use Xcode 10.2
2019-06-20 20:30:26 -07:00
brew "lz4"
GH-37923: [R] Move macOS build system to nixlibs.R (#37684) This PR modifies the build system of the R package to no longer rely on auto/homebrew. Instead this PR adds the infrastructure and code paths to use the same type of pre-compiled libarrow binaries as we use for Linux. The main difference is the use of the binaries even on CRAN (as we previously also used binaries in form of brew bottles). The addition of the new artifacts to tasks.yml should ensure that they get uploaded to the nightly repo as well as to the artifactory during the release (@ kou please confirm). A summary of the changes in this PR: - update `r/configure` and `r/tools/nixlibs.R` to enable the source build on macOS and usage of precompiled binaries using the existing mechanism to test compile a program to detect the exisitng openssl version - added tests for the changes in nixlibs.R - update the binary allow-list - Add the build jobs for libarrow binaries for arm64 and x86_64 macos with openssl 1.1 and 3.0 to the `r-binary-packages` job - Use the binaries to build the nightly packages - bump snappy version to 1.1.10 (and patch it on 10.13) due to build issues with the current version. This also touches on a number of issues in regards to a sanitizer issue we have had for a long time: #32562 #31766 - Disable the centos binary test step: #37922 Follow up issues: - #37921 - #37941 - #37945 * Closes: #37923 Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Co-authored-by: Jonathan Keane <jkeane@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
2023-10-05 14:48:44 +02:00
brew "mimalloc"
ARROW-5222: [Python] Revise pyarrow installation instructions for macOS This patch adds some docs and some links to previously existing documentation to help clarify the problems outlined on https://issues.apache.org/jira/browse/ARROW-5222. It also resolves the multiple-Brewfile issue by factoring out a common cpp/Brewfile. There are also some assorted formatting cleanups here too. Author: Neal Richardson <neal.p.richardson@gmail.com> Closes #4231 from nealrichardson/pyarrow-macos-dev-instructions and squashes the following commits: 716f2607f <Neal Richardson> Brewfile tweak 6014fac37 <Neal Richardson> Tweak language about conda on macos 13385853e <Neal Richardson> Update .gitignore 33a041c7f <Neal Richardson> Factor out a cpp/Brewfile and document it (thanks @kou) 9d333e517 <Neal Richardson> Use :ref: to link to other sections 4158d70be <Neal Richardson> Revise pyarrow installation instructions for macOS
2019-05-09 18:44:49 +02:00
brew "ninja"
GH-36886: [C++] Configure `azurite` in preparation for testing Azure C++ filesystem (#36988) ### Rationale for this change We need to write tests for https://github.com/apache/arrow/issues/18014. azurite is like a fake Azure blob storage so it can be used to write integration tests ### What changes are included in this PR? Extract the `azurite` related changes from https://github.com/apache/arrow/pull/12914 to create a smaller PR that's easier to review. I have made very minimal changes compared to that PR. Currently `azurite` is configured for all the environments where `ARROW_AZURE` was enabled by https://github.com/apache/arrow/pull/35701. I assume its deliberate that its not enabled yet for windows, alpine, conda, debian or fedora builds. ### Are these changes tested? Its tested by there aren't really any good tests in this PR. I used this `azurite` config in https://github.com/apache/arrow/pull/36835 to make an integration test that uses the Azure C++ SDK. On its own we can't really write tests for this `azurite` setup PR. ### Are there any user-facing changes? No * Closes: #36886 Lead-authored-by: Thomas Newton <thomas.w.newton@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
2023-08-07 21:48:29 +01:00
brew "node"
GH-45784: [C++] Unpin LLVM and OpenSSL in Brewfile (#45785) ### Rationale for this change We don't need to pin LLVM and OpenSSL now. ### What changes are included in this PR? Unpin LLVM and OpenSSL. ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: #45784 Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
2025-03-18 15:55:32 +09:00
brew "openssl"
GH-44797: [CI] Update pkg-config to pkgconf on Homebrew (#44798) ### Rationale for this change Some CI jobs are currently failing on main: - [Python / ARM64 macOS 14 Python 3](https://github.com/apache/arrow/actions/runs/11936033926/job/33268789966?pr=44796) - [C++ / ARM64 macOS 14 C++ ](https://github.com/apache/arrow/actions/runs/11935481942/job/33268873278) ### What changes are included in this PR? Update from deprecated `pkg-config` to `pkgconf` ### Are these changes tested? Via CI ### Are there any user-facing changes? No * GitHub Issue: #44797 Lead-authored-by: Raúl Cumplido <raulcumplido@gmail.com> Co-authored-by: Sutou Kouhei <kou@clear-code.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
2024-11-21 09:51:11 +01:00
brew "pkgconf"
GH-35987: [C++] Unpin brew protobuf version (#36087) Unpin brew protobuf version because of an upstream patch. * Closes: #35987 Authored-by: Jin Shang <shangjin1997@gmail.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
2023-06-16 14:55:02 +08:00
brew "protobuf"
ARROW-5660: [GLib][CI] Use Xcode 10.2 Author: Sutou Kouhei <kou@clear-code.com> Closes #4641 from kou/glib-ci-macos-10.2 and squashes the following commits: e7a926823 <Sutou Kouhei> Use Xcode 10.2
2019-06-20 20:30:26 -07:00
brew "python"
brew "rapidjson"
GH-37923: [R] Move macOS build system to nixlibs.R (#37684) This PR modifies the build system of the R package to no longer rely on auto/homebrew. Instead this PR adds the infrastructure and code paths to use the same type of pre-compiled libarrow binaries as we use for Linux. The main difference is the use of the binaries even on CRAN (as we previously also used binaries in form of brew bottles). The addition of the new artifacts to tasks.yml should ensure that they get uploaded to the nightly repo as well as to the artifactory during the release (@ kou please confirm). A summary of the changes in this PR: - update `r/configure` and `r/tools/nixlibs.R` to enable the source build on macOS and usage of precompiled binaries using the existing mechanism to test compile a program to detect the exisitng openssl version - added tests for the changes in nixlibs.R - update the binary allow-list - Add the build jobs for libarrow binaries for arm64 and x86_64 macos with openssl 1.1 and 3.0 to the `r-binary-packages` job - Use the binaries to build the nightly packages - bump snappy version to 1.1.10 (and patch it on 10.13) due to build issues with the current version. This also touches on a number of issues in regards to a sanitizer issue we have had for a long time: #32562 #31766 - Disable the centos binary test step: #37922 Follow up issues: - #37921 - #37941 - #37945 * Closes: #37923 Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Co-authored-by: Jonathan Keane <jkeane@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
2023-10-05 14:48:44 +02:00
brew "re2"
ARROW-5660: [GLib][CI] Use Xcode 10.2 Author: Sutou Kouhei <kou@clear-code.com> Closes #4641 from kou/glib-ci-macos-10.2 and squashes the following commits: e7a926823 <Sutou Kouhei> Use Xcode 10.2
2019-06-20 20:30:26 -07:00
brew "snappy"
ARROW-1662: Move to using Homebrew/bundle and Brewfile improves osx dependency management Author: Stephen G <stephen@egroat.com> Author: Stephen <stephengroat@users.noreply.github.com> Author: Stephen Groat <stephengroat@Stephens-MacBook-Pro.local> Closes #1143 from stephengroat/osx-brew and squashes the following commits: 81d4ac15 [Stephen G] Merge branch 'osx-brew' of https://github.com/stephengroat/arrow into osx-brew 62a79291 [Stephen G] Add license headers cee779b1 [Stephen] increase to max travis_wait 583ec0a7 [Stephen G] Move to individual brewfiles 65148557 [Stephen] add travis wait to get around timeouts 6faef0ce [Stephen Groat] Move to using Homebrew/bundle and Brewfile
2017-10-10 23:47:29 -04:00
brew "thrift"
GH-37923: [R] Move macOS build system to nixlibs.R (#37684) This PR modifies the build system of the R package to no longer rely on auto/homebrew. Instead this PR adds the infrastructure and code paths to use the same type of pre-compiled libarrow binaries as we use for Linux. The main difference is the use of the binaries even on CRAN (as we previously also used binaries in form of brew bottles). The addition of the new artifacts to tasks.yml should ensure that they get uploaded to the nightly repo as well as to the artifactory during the release (@ kou please confirm). A summary of the changes in this PR: - update `r/configure` and `r/tools/nixlibs.R` to enable the source build on macOS and usage of precompiled binaries using the existing mechanism to test compile a program to detect the exisitng openssl version - added tests for the changes in nixlibs.R - update the binary allow-list - Add the build jobs for libarrow binaries for arm64 and x86_64 macos with openssl 1.1 and 3.0 to the `r-binary-packages` job - Use the binaries to build the nightly packages - bump snappy version to 1.1.10 (and patch it on 10.13) due to build issues with the current version. This also touches on a number of issues in regards to a sanitizer issue we have had for a long time: #32562 #31766 - Disable the centos binary test step: #37922 Follow up issues: - #37921 - #37941 - #37945 * Closes: #37923 Lead-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de> Co-authored-by: Jonathan Keane <jkeane@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Signed-off-by: Dewey Dunnington <dewey@voltrondata.com>
2023-10-05 14:48:44 +02:00
brew "utf8proc"
ARROW-5222: [Python] Revise pyarrow installation instructions for macOS This patch adds some docs and some links to previously existing documentation to help clarify the problems outlined on https://issues.apache.org/jira/browse/ARROW-5222. It also resolves the multiple-Brewfile issue by factoring out a common cpp/Brewfile. There are also some assorted formatting cleanups here too. Author: Neal Richardson <neal.p.richardson@gmail.com> Closes #4231 from nealrichardson/pyarrow-macos-dev-instructions and squashes the following commits: 716f2607f <Neal Richardson> Brewfile tweak 6014fac37 <Neal Richardson> Tweak language about conda on macos 13385853e <Neal Richardson> Update .gitignore 33a041c7f <Neal Richardson> Factor out a cpp/Brewfile and document it (thanks @kou) 9d333e517 <Neal Richardson> Use :ref: to link to other sections 4158d70be <Neal Richardson> Revise pyarrow installation instructions for macOS
2019-05-09 18:44:49 +02:00
brew "wget"
ARROW-17081: [Java][Datasets] Move JNI build configuration from cpp/ to java/ (#13911) Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
2022-09-06 14:52:52 +09:00
brew "xsimd"
ARROW-5660: [GLib][CI] Use Xcode 10.2 Author: Sutou Kouhei <kou@clear-code.com> Closes #4641 from kou/glib-ci-macos-10.2 and squashes the following commits: e7a926823 <Sutou Kouhei> Use Xcode 10.2
2019-06-20 20:30:26 -07:00
brew "zstd"
Reference in New Issue Copy Permalink