Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow 64kb pagesize on aarch64 #1116

Merged
merged 1 commit into from
Jul 14, 2023
Merged

Conversation

benfred
Copy link
Contributor

@benfred benfred commented Jul 11, 2023

Closes #1115

Checklist

  • Used a personal fork of the feedstock to propose changes
  • Bumped the build number (if the version is unchanged)
  • Reset the build number to 0 (if the version changed)
  • Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
  • Ensured the license file is being packaged.

@conda-forge-webservices
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@benfred benfred marked this pull request as draft July 11, 2023 20:53
@jakirkham
Copy link
Member

jakirkham commented Jul 11, 2023

Thanks Ben! 🙏

Could you please add this to conda-forge.yml (relevant conda-forge docs)?

azure:
  store_build_artifacts: True

Then re-render to propagate that change to CI

This will cache the packages in CI so that people can try downloading and testing them

@benfred
Copy link
Contributor Author

benfred commented Jul 11, 2023

@conda-forge-admin, please rerender

@jakirkham
Copy link
Member

Thanks Ben! 🙏

Once CI completes it should cache the packages in the CI jobs

More details about how to retrieve them in this doc

@jakirkham
Copy link
Member

jakirkham commented Jul 12, 2023

Looks like there are now artifacts on CI

This covers nearly all jobs except for a couple with known issues ( #1114 )

@benfred benfred marked this pull request as ready for review July 12, 2023 04:40
@benfred
Copy link
Contributor Author

benfred commented Jul 12, 2023

after installing the CI artifacts here on my aarch64 system w/ 64Kb pages - I'm no longer seeing the jemalloc error:

In [1]: import pyarrow

In [2]: pyarrow
Out[2]: <module 'pyarrow' from '/opt/conda/envs/rapids/lib/python3.10/site-packages/pyarrow/__init__.py'>

@jakirkham
Copy link
Member

@xhochy would you be interested in testing these on macOS ARM to confirm they are working ok there?

@jakirkham
Copy link
Member

Am on MacBook Pro (16-inch, 2021) with an Apple M1 Pro chip


Tried downloading the CI artifact for macOS ARM (named arrow-cpp-feedstock_conda_artifacts_20230711.4.1_osx_arm64_.zip) and created a new environment with it:

$ conda list
# packages in environment at /Users/jkirkham/miniforge/envs/tst_pyarrow:
#
# Name                    Version                   Build  Channel
appnope                   0.1.3              pyhd8ed1ab_0    conda-forge
asttokens                 2.2.1              pyhd8ed1ab_0    conda-forge
aws-c-auth                0.7.0                hf4e22b4_2    conda-forge
aws-c-cal                 0.6.0                h43a37b4_0    conda-forge
aws-c-common              0.8.23               hb547adb_0    conda-forge
aws-c-compression         0.2.17               h31a6291_1    conda-forge
aws-c-event-stream        0.3.1                h1e1aaf7_1    conda-forge
aws-c-http                0.7.11               h3107daf_0    conda-forge
aws-c-io                  0.13.28              hf096fb3_0    conda-forge
aws-c-mqtt                0.8.14               h9418dbe_2    conda-forge
aws-c-s3                  0.3.13               h79b0d72_2    conda-forge
aws-c-sdkutils            0.1.11               h31a6291_1    conda-forge
aws-checksums             0.1.16               h31a6291_1    conda-forge
aws-crt-cpp               0.20.3               h0918d87_4    conda-forge
aws-sdk-cpp               1.10.57             h6f3a27c_17    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
bzip2                     1.0.8                h3422bc3_4    conda-forge
c-ares                    1.19.1               hb547adb_0    conda-forge
ca-certificates           2023.5.7             hf0a4a13_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
executing                 1.2.0              pyhd8ed1ab_0    conda-forge
gflags                    2.2.2             hc88da5d_1004    conda-forge
glog                      0.6.0                h6da1cb0_0    conda-forge
ipython                   8.14.0             pyhd1c38e8_0    conda-forge
jedi                      0.18.2             pyhd8ed1ab_0    conda-forge
krb5                      1.20.1               h69eda48_0    conda-forge
libabseil                 20230125.3      cxx17_h13dd4ca_0    conda-forge
libarrow                  12.0.1           hdeb1470_5_cpu    file:///Users/jkirkham/Downloads/arrow-cpp-feedstock_conda_artifacts_20230711
libblas                   3.9.0           17_osxarm64_openblas    conda-forge
libbrotlicommon           1.0.9                h1a8c8d9_9    conda-forge
libbrotlidec              1.0.9                h1a8c8d9_9    conda-forge
libbrotlienc              1.0.9                h1a8c8d9_9    conda-forge
libcblas                  3.9.0           17_osxarm64_openblas    conda-forge
libcrc32c                 1.1.2                hbdafb3b_0    conda-forge
libcurl                   8.1.2                h912dcd9_0    conda-forge
libcxx                    16.0.6               h4653b0c_0    conda-forge
libedit                   3.1.20191231         hc8eb9b7_2    conda-forge
libev                     4.33                 h642e427_1    conda-forge
libevent                  2.1.12               h2757513_1    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libgfortran               5.0.0           12_2_0_hd922786_31    conda-forge
libgfortran5              12.2.0              h0eea778_31    conda-forge
libgoogle-cloud           2.12.0               h05652e3_1    conda-forge
libgrpc                   1.56.1               h9075ed4_0    conda-forge
liblapack                 3.9.0           17_osxarm64_openblas    conda-forge
libnghttp2                1.52.0               hae82a92_0    conda-forge
libopenblas               0.3.23          openmp_hc731615_0    conda-forge
libprotobuf               4.23.3               hf32f9b9_0    conda-forge
libsqlite                 3.42.0               hb31c410_0    conda-forge
libssh2                   1.11.0               h7a5bd25_0    conda-forge
libthrift                 0.18.1               ha061701_2    conda-forge
libutf8proc               2.8.0                h1a8c8d9_0    conda-forge
libzlib                   1.2.13               h53f4e23_5    conda-forge
llvm-openmp               16.0.6               h1c12783_0    conda-forge
lz4-c                     1.9.4                hb7217d7_0    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
ncurses                   6.4                  h7ea286d_0    conda-forge
numpy                     1.25.1          py310haa1e00c_0    conda-forge
openssl                   3.1.1                h53f4e23_1    conda-forge
orc                       1.9.0                h858f345_1    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pexpect                   4.8.0              pyh1a96a4e_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pip                       23.1.2             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.39             pyha770c72_0    conda-forge
prompt_toolkit            3.0.39               hd8ed1ab_0    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pyarrow                   12.0.1          py310hfbab16f_5_cpu    file:///Users/jkirkham/Downloads/arrow-cpp-feedstock_conda_artifacts_20230711
pygments                  2.15.1             pyhd8ed1ab_0    conda-forge
python                    3.10.12         h01493a6_0_cpython    conda-forge
python_abi                3.10                    3_cp310    conda-forge
re2                       2023.03.02           hc5e2d97_0    conda-forge
readline                  8.2                  h92ec313_1    conda-forge
setuptools                68.0.0             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
snappy                    1.1.10               h17c5cce_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
tk                        8.6.12               he1e0b03_0    conda-forge
traitlets                 5.9.0              pyhd8ed1ab_0    conda-forge
typing_extensions         4.7.1              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
wcwidth                   0.2.6              pyhd8ed1ab_0    conda-forge
wheel                     0.40.0             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h57fd34a_0    conda-forge
zstd                      1.5.2                h4f39d0f_7    conda-forge

In this environment tried to import pyarrow (as seen below) and it appears to work:

$ ipython
Python 3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:41:52) [Clang 15.0.7 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.14.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pyarrow

In [2]: pyarrow
Out[2]: <module 'pyarrow' from '/Users/jkirkham/miniforge/envs/tst_pyarrow/lib/python3.10/site-packages/pyarrow/__init__.py'>

@h-vetinari
Copy link
Member

Thanks for the PR and the testing effort! I'm happy to wait for @xhochy's feedback, but I also think the PR makes sense and having been tested twice is already very good.

Also, CI everywhere else is passing, so if you forcepush back to 8cd448c, then I'd be happy to merge this (and backport it to 11.x and 10.x)

@benfred benfred force-pushed the 64k_pagesize branch 2 times, most recently from abaab94 to 8cd448c Compare July 12, 2023 20:15
@benfred
Copy link
Contributor Author

benfred commented Jul 12, 2023

thanks @h-vetinari ! I've force pushed back to 8cd448c

Copy link
Member

@h-vetinari h-vetinari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, if you want to have a look @xhochy; otherwise I'll merge in 24h :)

benfred added a commit to benfred/arrow-cpp-feedstock that referenced this pull request Jul 13, 2023
@h-vetinari h-vetinari merged commit 3339a3f into conda-forge:main Jul 14, 2023
8 of 11 checks passed
@h-vetinari h-vetinari mentioned this pull request Jul 14, 2023
5 tasks
benfred added a commit to benfred/arrow-cpp-feedstock that referenced this pull request Jul 14, 2023
benfred added a commit to benfred/arrow-cpp-feedstock that referenced this pull request Jul 14, 2023
@benfred benfred deleted the 64k_pagesize branch July 14, 2023 17:12
benfred added a commit to benfred/arrow-cpp-feedstock that referenced this pull request Jul 14, 2023
h-vetinari added a commit that referenced this pull request Jul 14, 2023
github-actions bot added a commit that referenced this pull request Jul 14, 2023
automerged PR by conda-forge/automerge-action
github-actions bot added a commit that referenced this pull request Jul 14, 2023
automerged PR by conda-forge/automerge-action
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error on systems with a pagesize > 16KB
3 participants