Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: build sdist and wheels (Linux & MacOS) #35

Merged
merged 44 commits into from
Aug 21, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
a8b5453
ci: build wheels (wip)
benbovy Aug 19, 2024
e1205d4
disable running cmake for sdist
benbovy Aug 19, 2024
338a850
sdist: add missing --sdist option
benbovy Aug 19, 2024
47ac72f
fix 3rd-party directory
benbovy Aug 19, 2024
8f06f8f
fix s2geography version
benbovy Aug 19, 2024
81ab7a6
fix s2geography tag
benbovy Aug 19, 2024
76d355d
fix ci install bash script permission
benbovy Aug 19, 2024
9ea9fd4
fix install script fn name
benbovy Aug 19, 2024
00b5189
typo
benbovy Aug 19, 2024
e1bc696
download dependencies in build script
benbovy Aug 19, 2024
effae57
build script: cd into root dependencies dir
benbovy Aug 19, 2024
5c91308
fix tar?
benbovy Aug 19, 2024
2cf32e0
fix deps download hopefully
benbovy Aug 19, 2024
0d87d92
create source directory before download
benbovy Aug 19, 2024
2be2923
typo
benbovy Aug 19, 2024
0472c4b
typos and old cmake behavior for cmake _ROOT
benbovy Aug 19, 2024
23e77ee
openssl (header files) seems missing
benbovy Aug 20, 2024
3a3b676
revert previous commit, set OPENSSL_ROOT_DIR on linux
benbovy Aug 20, 2024
ff3c458
download & build openssl in dependencies script
benbovy Aug 20, 2024
ff4b2df
revert building openssl, try ubuntu-latest
benbovy Aug 20, 2024
4389f31
try force reinstall openssl
benbovy Aug 20, 2024
f76a522
install openssl in linux container using yum
benbovy Aug 20, 2024
40d52cf
typo
benbovy Aug 20, 2024
2b4e9ef
make installed deps discoverable more globally
benbovy Aug 20, 2024
20de9a9
try fix openssl transitive headers not found
benbovy Aug 20, 2024
7b14ff6
another attempt at fixing openssl header lookup
benbovy Aug 20, 2024
b48ff60
repair wheel: add 3rd-party install path
benbovy Aug 20, 2024
6943800
update s2geography patch
benbovy Aug 20, 2024
8437f66
also require openssl in spherely
benbovy Aug 20, 2024
15d9382
audithwheel command not found?? -> typo!
benbovy Aug 20, 2024
65c6220
auditwheel not on macOS
benbovy Aug 20, 2024
9703a57
typo
benbovy Aug 20, 2024
06ee3fb
more logs
benbovy Aug 20, 2024
28f3eb1
fix wheel repair on linux and macos?
benbovy Aug 20, 2024
5515c9c
repair wheels: use DYLD_LIBRARY_PATH on macos
benbovy Aug 20, 2024
8c1117d
bump minimum required python version to 3.10
benbovy Aug 20, 2024
1d763ca
fix pytest tests location?
benbovy Aug 20, 2024
07df1bc
bump macos deployment target
benbovy Aug 21, 2024
dce651c
fix pytest command?
benbovy Aug 21, 2024
1ea8e31
update patch (upstream ci fix suggestion)
benbovy Aug 21, 2024
43b7ce0
add upload step
benbovy Aug 21, 2024
fc7cf50
revert last change in patch
benbovy Aug 21, 2024
cf99795
add 3rd-party licenses in the built wheels
benbovy Aug 21, 2024
faa6bec
add ci job upload release files to pypi
benbovy Aug 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
133 changes: 133 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
name: Build and publish

on:
# trigger action from GitHub GUI (testing, no publish)
workflow_dispatch:
release:
types:
- published
pull_request: # also build on PRs touching this file
paths:
- ".github/workflows/release.yml"
- "MANIFEST.in"
- "pyproject.toml"
- "setup.py"

jobs:
build_sdist:
name: Build sdist
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Build SDist
run: pipx run build

- uses: actions/upload-artifact@v4
with:
name: release-sdist
path: ./dist/*

- name: Check metadata
run: pipx run twine check dist/*

build_wheels:
name: Build binary wheel on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
env:
ABSL_VERSION: "20240722.0"
S2GEOMETRY_VERSION: "0.11.1"
S2GEOGRAPHY_VERSION: "0.2.0"
CXX_STANDARD: "17"
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-20.04
arch: x86_64
# - os: windows-2019
# arch: x86
# msvc_arch: x86
# - os: windows-2019
# arch: AMD64
# msvc_arch: x64
- os: macos-13
arch: x86_64
cmake_osx_architectures: x86_64
- os: macos-14
arch: arm64
cmake_osx_architectures: arm64

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Cache 3rd-party Directory
id: cache-build
uses: actions/cache@v4
with:
path: ${{ runner.temp }}/3rd-party
key: ${{ matrix.os }}-${{ matrix.arch }}-${{ env.ABSL_VERSION }}-${{ env.S2GEOMETRY_VERSION }}-${{ env.S2GEOGRAPHY_VERSION }}-${{ hashFiles('ci/*') }}

- name: Fetch absl
if: steps.cache-build.outputs.cache-hit != 'true'
uses: actions/checkout@v4
with:
repository: abseil/abseil-cpp
ref: ${{ env.ABSL_VERSION }}
path: ${{ runner.temp }}/3rd-party/absl-src-${{ env.ABSL_VERSION }}
benbovy marked this conversation as resolved.
Show resolved Hide resolved
fetch-depth: 0

- name: Fetch s2geometry
if: steps.cache-build.outputs.cache-hit != 'true'
uses: actions/checkout@v4
with:
repository: google/s2geometry
ref: v${{ env.S2GEOMETRY_VERSION }}
path: ${{ runner.temp }}/3rd-party/s2geometry-src-${{ env.S2GEOMETRY_VERSION }}
fetch-depth: 0

- name: Fetch s2geography
if: steps.cache-build.outputs.cache-hit != 'true'
uses: actions/checkout@v4
with:
repository: paleolimbot/s2geography
ref: v${{ env.S2GEOGRAPHY_VERSION }}
path: ${{ runner.temp }}/3rd-party/s2geography-src-${{ env.S2GEOGRAPHY_VERSION }}
fetch-depth: 0

- name: Build wheels
uses: pypa/[email protected]
env:
CIBW_ARCHS: ${{ matrix.arch }}
CIBW_SKIP: cp36-* pp* *musllinux* *-manylinux_i686
CIBW_TEST_SKIP: "cp38-macosx_arm64"
CIBW_ENVIRONMENT_LINUX:
DEPENDENCIES_DIR=${{ runner.temp }}/3rd-party
ABSL_VERSION=${{ env.ABSL_VERSION }}
S2GEOMETRY_VERSION=${{ env.S2GEOMETRY_VERSION }}
S2GEOGRAPHY_VERSION=${{ env.S2GEOGRAPHY_VERSION }}
CXXSTANDARD=${{ env.CXXSTANDARD }}
CIBW_ENVIRONMENT_MACOS:
DEPENDENCIES_DIR=${{ runner.temp }}/3rd-party
ABSL_VERSION=${{ env.ABSL_VERSION }}
S2GEOMETRY_VERSION=${{ env.S2GEOMETRY_VERSION }}
S2GEOGRAPHY_VERSION=${{ env.S2GEOGRAPHY_VERSION }}
CXXSTANDARD=${{ env.CXXSTANDARD }}
MACOSX_DEPLOYMENT_TARGET=10.9
CMAKE_OSX_ARCHITECTURES='${{ matrix.cmake_osx_architectures }}'
CIBW_ENVIRONMENT_WINDOWS:
DEPENDENCIES_DIR=${{ runner.temp }}/3rd-party
ABSL_VERSION=${{ env.ABSL_VERSION }}
S2GEOMETRY_VERSION=${{ env.S2GEOMETRY_VERSION }}
S2GEOGRAPHY_VERSION=${{ env.S2GEOGRAPHY_VERSION }}
CXXSTANDARD=${{ env.CXXSTANDARD }}
CIBW_BEFORE_ALL: ./ci/install_3rdparty.sh
CIBW_BEFORE_ALL_WINDOWS: ci\install_geos.cmd
# CIBW_BEFORE_BUILD_WINDOWS: pip install delvewheel
# CIBW_REPAIR_WHEEL_COMMAND_WINDOWS: delvewheel repair --add-path ${{ runner.temp }}\geos-${{ env.GEOS_VERSION }}\bin -w {dest_dir} {wheel}
CIBW_TEST_REQUIRES: pytest
CIBW_TEST_COMMAND: pytest .
102 changes: 102 additions & 0 deletions ci/install_3rdparty.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
#!/bin/bash

# Build and install absl, s2 and s2geography on posix systems.
#
# This script requires environment variables to be set
# - DEPENDENCIES_DIR=/path/to/cached/prefix -- to build or use as cache
# - ABSL_VERSION
# - S2GEOMETRY_VERSION
# - S2GEOGRAPHY_VERSION
# - CXXSTANDARD
#
# This script assumes that library sources have been downloaded or copied in
# DEPENDENCIES_DIR (e.g., $DEPENDENCIES_DIR/absl-src-$ABSL_VERSION).
pushd .

set -e

if [ -z "$DEPENDENCIES_DIR" ]; then
echo "DEPENDENCIES_DIR must be set"
exit 1
elif [ -z "$ABSL_VERSION" ]; then
echo "ABSL_VERSION must be set"
exit 1
elif [ -z "$S2GEOMETRY_VERSION" ]; then
echo "S2GEOMETRY_VERSION must be set"
exit 1
elif [ -z "$S2GEOGRAPHY_VERSION" ]; then
echo "S2GEOGRAPHY_VERSION must be set"
exit 1
fi

BUILD_DIR=$DEPENDENCIES_DIR/build
INSTALL_DIR=$DEPENDENCIES_DIR/dist

mkdir -p $BUILD_DIR
mkdir -p $INSTALL_DIR

ABSL_SRC_DIR=$DEPENDENCIES/absl-src-$ABSL_VERSION
S2GEOMETRY_SRC_DIR=$DEPENDENCIES/s2geometry-src-$S2GEOMETRY_VERSION
S2GEOGRAPHY_SRC_DIR=$DEPENDENCIES/s2geography-src-$S2GEOGRAPHY_VERSION

ABSL_BUILD_DIR=$BUILD_DIR/absl-src-$ABSL_VERSION
S2GEOMETRY_BUILD_DIR=$BUILD_DIR/s2geometry-src-$S2GEOMETRY_VERSION
S2GEOGRAPHY_BUILD_DIR=$BUILD_DIR/s2geography-src-$S2GEOGRAPHY_VERSION

build_3rd_party(){
echo "Installing cmake"
pip install cmake

rm -rf $BUILD_DIR
rm -rf $INSTALL_DIR

echo "Building and installing absl-$ABSL_VERSION"

cmake -S $ABSL_SRC_DIR -B $ABSL_BUILD_DIR \
-DCMAKE_INSTALL_PREFIX=$INSTALL_DIR \
-DCMAKE_POSITION_INDEPENDENT_CODE=ON \
-DCMAKE_CXX_STANDARD=$CXX_STANDARD \
-DCMAKE_BUILD_TYPE=Release \
-DABSL_ENABLE_INSTALL=ON

cmake --build $ABSL_BUILD_DIR
cmake --install $ABSL_BUILD_DIR

echo "Building and installing s2geometry-$S2GEOMETRY_VERSION"

cmake -S $S2GEOMETRY_SRC_DIR -B $S2GEOMETRY_BUILD_DIR \
-DCMAKE_INSTALL_PREFIX=$INSTALL_DIR \
-DABSL_ROOT=$INSTALL_DIR \
-DBUILD_TESTS=OFF \
-DBUILD_EXAMPLES=OFF \
-UGOOGLETEST_ROOT \
-DCMAKE_CXX_STANDARD=$CXX_STANDARD \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_SHARED_LIBS=ON

cmake --build $S2GEOMETRY_BUILD_DIR
cmake --install $S2GEOMETRY_BUILD_DIR

echo "Building and installing s2geography-$S2GEOGRAPHY_VERSION"

cmake -S $S2GEOGRAPHY_SRC_DIR -B $S2GEOGRAPHY_BUILD_DIR \
-DCMAKE_INSTALL_PREFIX=$INSTALL_DIR \
-DABSL_ROOT=$INSTALL_DIR \
-DS2_ROOT=$INSTALL_DIR \
-DBUILD_TESTS=OFF \
-DS2GEOGRAPHY_S2_SOURCE=SYSTEM \
-DBUILD_EXAMPLES=OFF \
-DCMAKE_CXX_STANDARD=$CXX_STANDARD \
-DBUILD_SHARED_LIBS=ON

cmake --build $S2GEOGRAPHY_BUILD_DIR
cmake --install $S2GEOGRAPHY_BUILD_DIR
}

if [ -d "$INSTALL_DIR/include/s2geography" ]; then
echo "Using cached install directory $INSTALL_DIR"
else
build_geos
fi

popd
4 changes: 4 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ Repository = "https://github.com/benbovy/spherely"
[project.optional-dependencies]
test = ["pytest>=6.0"]

[tool.scikit-build]
sdist.exclude = [".github"]
sdist.cmake = false

[tool.mypy]
files = ["tests", "src/spherely.pyi"]
show_error_codes = true
Expand Down
Loading