2016-11-11 14:18:09 -05:00
.. Licensed to the Apache Software Foundation (ASF) under one
.. or more contributor license agreements. See the NOTICE file
.. distributed with this work for additional information
.. regarding copyright ownership. The ASF licenses this file
.. to you under the Apache License, Version 2.0 (the
.. "License"); you may not use this file except in compliance
.. with the License. You may obtain a copy of the License at
.. http://www.apache.org/licenses/LICENSE-2.0
.. Unless required by applicable law or agreed to in writing,
.. software distributed under the License is distributed on an
.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
.. KIND, either express or implied. See the License for the
.. specific language governing permissions and limitations
.. under the License.
2018-12-19 15:52:23 +01:00
Installing PyArrow
==================
2016-11-11 14:18:09 -05:00
2019-02-11 19:30:22 +01:00
System Compatibility
--------------------
2016-11-11 14:18:09 -05:00
2023-03-27 10:07:42 +09:00
PyArrow is regularly built and tested on Windows, macOS and various
Linux distributions. We strongly recommend using a 64-bit system.
2019-02-11 19:30:22 +01:00
Python Compatibility
--------------------
2025-10-21 11:54:27 +02:00
PyArrow is currently compatible with Python 3.10, 3.11, 3.12, 3.13 and 3.14.
2019-02-11 19:30:22 +01:00
Using Conda
-----------
Install the latest version of PyArrow from
`conda-forge <https://conda-forge.org/> `_ using `Conda <https://conda.io> `_ :
2016-11-11 14:18:09 -05:00
.. code-block :: bash
conda install -c conda-forge pyarrow
2024-05-16 15:20:47 -08:00
.. note ::
While the `` pyarrow `` `conda-forge <https://conda-forge.org/> `_ package is
the right choice for most users, both a minimal and maximal variant of the
package exist, either of which may be better for your use case. See
:ref: `python-conda-differences` .
2019-02-11 19:30:22 +01:00
Using Pip
---------
2016-11-11 14:18:09 -05:00
2019-02-11 19:30:22 +01:00
Install the latest version from `PyPI <https://pypi.org/> `_ (Windows, Linux,
and macOS):
2016-11-11 14:18:09 -05:00
.. code-block :: bash
2026-01-30 09:09:33 +01:00
pip install pyarrow
2016-11-11 14:18:09 -05:00
2017-09-19 23:46:15 -04:00
If you encounter any importing issues of the pip wheels on Windows, you may
2025-09-01 15:33:27 +02:00
need to install the `latest Visual C++ Redistributable for Visual Studio
<https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170#latest-microsoft-visual-c-redistributable-version> `_.
2017-09-19 23:46:15 -04:00
2021-02-17 22:51:07 +01:00
.. warning ::
On Linux, you will need pip >= 19.0 to detect the prebuilt binary packages.
2023-10-05 13:56:15 +02:00
Installing nightly packages or from source
------------------------------------------
2016-11-11 14:18:09 -05:00
2019-03-17 16:26:34 -05:00
See :ref: `python-development` .
2022-11-22 11:54:38 +01:00
Dependencies
------------
Optional dependencies
2025-09-09 09:44:55 +02:00
* **NumPy 1.21.2** or higher.
* **pandas 1.3.4** or higher,
2022-11-22 11:54:38 +01:00
* **cffi** .
Additional packages PyArrow is compatible with are :ref: `fsspec <filesystem-fsspec>`
and **pytz** , **dateutil** or **tzdata** package for timezones.
2023-10-10 13:21:55 +02:00
tzdata on Windows
^^^^^^^^^^^^^^^^^
2026-03-03 14:19:25 +01:00
On Linux and macOS, Arrow uses the OS-provided timezone database. On Windows,
Arrow uses the Windows timezone database when built with MSVC or recent MinGW GCC
(version 13+), which covers most pre-built packages. No additional setup is needed
for these builds.
However, when PyArrow is built with Clang/libc++ on Windows, a user-provided
IANA timezone database is required. To download and extract the text version of
2023-10-10 13:21:55 +02:00
the IANA timezone database follow the instructions in the C++
2026-03-03 14:19:25 +01:00
:ref: `download-timezone-database` or use the (deprecated) pyarrow utility function
`` pyarrow.util.download_tzdata_on_windows() `` .
2023-10-10 13:21:55 +02:00
By default, the timezone database will be detected at `` %USERPROFILE%\Downloads\tzdata `` .
If the database has been downloaded in a different location, you will need to set
2026-03-03 14:19:25 +01:00
a custom path to the database from Python using the (deprecated)
`` pa.set_timezone_db_path("custom_path") `` function.
2023-10-10 13:21:55 +02:00
2026-03-03 14:19:25 +01:00
.. note ::
You may encounter problems writing datetime data to an ORC file if you install
pyarrow with pip. One possible solution to fix this problem:
2025-05-30 03:42:12 -04:00
1. Install tzdata with `` pip install tzdata ``
2. Set the environment variable `` TZDIR = path\to\.venv\Lib\site-packages\tzdata\ ``
2026-03-03 14:19:25 +01:00
You can find where `` tzdata `` is installed with the following python command:
2025-05-30 03:42:12 -04:00
2026-03-03 14:19:25 +01:00
.. code-block :: python
2025-05-30 03:42:12 -04:00
2026-03-03 14:19:25 +01:00
import tzdata
print ( tzdata . __file__ ) # path\to\.venv\Lib\site-packages\tzdata\__init__.py
2025-05-30 03:42:12 -04:00
2024-05-16 15:20:47 -08:00
.. _python-conda-differences:
Differences between conda-forge packages
----------------------------------------
On `conda-forge <https://conda-forge.org/> `_ , PyArrow is published as three
separate packages, each providing varying levels of functionality. This is in
contrast to PyPi, where only a single PyArrow package is provided.
The purpose of this split is to minimize the size of the installed package for
most users (`` pyarrow `` ), provide a smaller, minimal package for specialized use
cases (`` pyarrow-core `` ), while still providing a complete package for users who
require it (`` pyarrow-all `` ). What was historically `` pyarrow `` on
`conda-forge <https://conda-forge.org/> `_ is now `` pyarrow-all `` , though most
users can continue using `` pyarrow `` .
The `` pyarrow-core `` package includes the following functionality:
- :ref: `data`
- :ref: `compute` (i.e., `` pyarrow.compute `` )
- :ref: `io`
- :ref: `ipc` (i.e., `` pyarrow.ipc `` )
- :ref: `filesystem` (i.e., `` pyarrow.fs `` . Note: It's planned to move cloud fileystems (i.e., :ref: `S3<filesystem-s3>` , :ref: `GCS<filesystem-gcs>` , etc) into `` pyarrow `` in a future release though :ref: `filesystem-localfs` will remain in `` pyarrow-core `` .)
- File formats: :ref: `Arrow/Feather<feather>` , :ref: `JSON<json>` , :ref: `CSV<py-csv>` , :ref: `ORC<orc>` (but not Parquet)
The `` pyarrow `` package adds the following:
- Acero (i.e., `` pyarrow.acero `` )
- :ref: `dataset` (i.e., `` pyarrow.dataset `` )
- :ref: `Parquet<parquet>` (i.e., `` pyarrow.parquet `` )
- Substrait (i.e., `` pyarrow.substrait `` )
Finally, `` pyarrow-all `` adds:
- :ref: `flight` and Flight SQL (i.e., `` pyarrow.flight `` )
- Gandiva (i.e., `` pyarrow.gandiva `` )
The following table lists the functionality provided by each package and may be
useful when deciding to use one package over another or when
:ref: `python-conda-custom-selection` .
+------------+---------------------+--------------+---------+-------------+
| Component | Package | pyarrow-core | pyarrow | pyarrow-all |
+------------+---------------------+--------------+---------+-------------+
| Core | pyarrow-core | ✓ | ✓ | ✓ |
+------------+---------------------+--------------+---------+-------------+
| Parquet | libparquet | | ✓ | ✓ |
+------------+---------------------+--------------+---------+-------------+
| Dataset | libarrow-dataset | | ✓ | ✓ |
+------------+---------------------+--------------+---------+-------------+
| Acero | libarrow-acero | | ✓ | ✓ |
+------------+---------------------+--------------+---------+-------------+
| Substrait | libarrow-substrait | | ✓ | ✓ |
+------------+---------------------+--------------+---------+-------------+
| Flight | libarrow-flight | | | ✓ |
+------------+---------------------+--------------+---------+-------------+
| Flight SQL | libarrow-flight-sql | | | ✓ |
+------------+---------------------+--------------+---------+-------------+
| Gandiva | libarrow-gandiva | | | ✓ |
+------------+---------------------+--------------+---------+-------------+
.. _python-conda-custom-selection:
Creating A Custom Selection
^^^^^^^^^^^^^^^^^^^^^^^^^^^
If you know which components you need and want to control what's installed, you
can create a custom selection of packages to include only the extra features you
need. For example, to install `` pyarrow-core `` and add support for reading and
writing Parquet, install `` libparquet `` alongside `` pyarrow-core `` :
.. code-block :: shell
conda install -c conda-forge pyarrow-core libparquet
Or if you wish to use `` pyarrow `` but need support for Flight RPC:
.. code-block :: shell
conda install -c conda-forge pyarrow libarrow-flight
