Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sisl om build #1

Open
wants to merge 22 commits into
base: stable/v8.x
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .codecov.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,6 @@ codecov:
notify:
require_ci_to_pass: no

fixes:
- "deps/sisl/::"

ignore:
- "**/*_test.c*"
- "**/*_test.h*"
Expand Down
57 changes: 57 additions & 0 deletions .github/actions/load_conan/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: 'Load Conan Cache'
description: 'Loads Local Conan Cache'
inputs:
testing:
description: 'Support building tests'
required: true
key_prefix:
description: 'Cache prefix'
required: true
default: 'Deps'
fail_on_cache_miss:
description: 'Fail if key missing'
required: false
default: false
path:
description: 'Recipe path'
required: false
default: '.'
load_any:
description: 'Load cache miss'
required: false
default: 'False'
outputs:
cache-hit:
description: 'Cache match found'
value: ${{ steps.restore-cache.outputs.cache-hit }}
runs:
using: "composite"
steps:
- id: hash-key-primary
shell: bash
run: |
echo "key=${{ inputs.path }}/conanfile.py" >> $GITHUB_OUTPUT

- id: hash-key-3rd
shell: bash
run: |
echo "keys=${{ inputs.path }}/3rd_party/**/conanfile.py" >> $GITHUB_OUTPUT

- name: Restore Cache
id: restore-cache
uses: actions/cache/restore@v3
with:
path: |
~/.conan/data
key: ${{ inputs.key_prefix }}-${{ hashFiles(steps.hash-key-primary.outputs.key, steps.hash-key-3rd.outputs.keys) }}
fail-on-cache-miss: ${{ inputs.fail_on_cache_miss }}

- name: Restore Testing Cache
uses: actions/cache/restore@v3
with:
path: |
~/.conan/data
key: ${{ inputs.key_prefix }}-${{ hashFiles(steps.hash-key-primary.outputs.key, steps.hash-key-3rd.outputs.keys) }}
restore-keys: ${{ inputs.key_prefix }}-
if: ${{ steps.restore-cache.outputs.cache-hit != 'true' && (( github.event_name == 'pull_request' && inputs.testing == 'True' ) || ( inputs.load_any == 'True' )) }}

31 changes: 31 additions & 0 deletions .github/actions/setup_conan/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: 'Setup Conan'
description: 'Sets up Conan for Sisl Builds'
inputs:
platform:
description: 'Platform conan will be building on'
required: true
default: 'ubuntu-22.04'
runs:
using: "composite"
steps:
- name: Setup Python
uses: actions/setup-python@v3
with:
python-version: "3.8"

- name: Setup Conan and Export Recipes
shell: bash
run: |
python -m pip install --upgrade pip
python -m pip install conan~=1.0
python -m pip install gcovr
conan user
conan profile new --detect default

- name: Fixup libstdc++
shell: bash
run: |
# Set std::string to non-CoW C++11 version
sed -i 's,compiler.libcxx=libstdc++$,compiler.libcxx=libstdc++11,g' ~/.conan/profiles/default
if: ${{ inputs.platform == 'ubuntu-22.04' }}

36 changes: 36 additions & 0 deletions .github/actions/store_conan/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: 'Store Conan Cache'
description: 'Cleans Local Conan Cache and Persists Dirty Packages'
inputs:
key_prefix:
description: 'Cache prefix'
required: true
default: 'Deps'
runs:
using: "composite"
steps:
- name: Setup Conan and Export Recipes
shell: bash
run: |
if [ -d 3rd_party ]; then
dep_pkgs=$(ls -1d 3rd_party/* 2>/dev/null | cut -d'/' -f2 | paste -sd'|' - -)
fi
if [ -z "${dep_pkgs}" ]; then
dep_pkgs="no_3rd_party"
fi
dirty_pkgs=$(ls -1d ~/.conan/data/*/*/*/*/build 2>/dev/null | sed 's,.*data/,,')
if [ -z "${dirty_pkgs}" ]; then
dirty_pkgs="no_public/0"
fi
dirty_pkgs_d=$(echo "${dirty_pkgs}" | cut -d'/' -f1 | paste -sd'|' - -)
echo "::info:: Caching: ${dirty_pkgs_d}|${dep_pkgs}"
ls -1d ~/.conan/data/* | grep -Ev "(${dirty_pkgs_d}|${dep_pkgs})" | xargs rm -rf
rm -rf ~/.conan/data/*/*/*/*/build
rm -rf ~/.conan/data/*/*/*/*/source

- name: Save Cache
uses: actions/cache/save@v3
with:
path: |
~/.conan/data
key: ${{ inputs.key_prefix }}-${{ hashFiles('conanfile.py', '3rd_party/**/conanfile.py') }}

137 changes: 50 additions & 87 deletions .github/workflows/build_dependencies.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,62 +75,40 @@ jobs:
- name: Retrieve Code
uses: actions/checkout@v3
with:
path: deps/sisl
ref: ${{ inputs.branch }}
if: ${{ inputs.testing == 'True' }}

- name: Retrieve Recipe
uses: actions/checkout@v3
with:
repository: ebay/sisl
path: deps/sisl
repository: eBay/sisl
ref: ${{ inputs.branch }}
if: ${{ inputs.testing == 'False' }}

- name: Restore Sisl Cache
id: restore-cache-sisl
uses: actions/cache/restore@v3
- name: Load Conan Cache
id: restore-cache
uses: eBay/sisl/.github/actions/load_conan@stable/v8.x
with:
path: |
~/.conan/data
key: SislDeps8-${{ inputs.platform }}-${{ inputs.build-type }}-${{ inputs.malloc-impl }}-${{ inputs.prerelease }}-${{ hashFiles('**/conanfile.py') }}
testing: ${{ inputs.testing }}
key_prefix: SislDeps8-${{ inputs.platform }}-${{ inputs.build-type }}-${{ inputs.malloc-impl }}-${{ inputs.prerelease }}

- name: Restore Testing Cache
id: restore-cache-testing-sisl
uses: actions/cache/restore@v3
- name: Setup Conan
uses: eBay/sisl/.github/actions/setup_conan@stable/v8.x
with:
path: |
~/.conan/data
key: SislDeps8-${{ inputs.platform }}-${{ inputs.build-type }}-${{ inputs.malloc-impl }}-${{ inputs.prerelease }}-
if: ${{ github.event_name == 'pull_request' && inputs.testing == 'True' && steps.restore-cache-sisl.outputs.cache-hit != 'true' }}
platform: ${{ inputs.platform }}
if: ${{ inputs.testing == 'True' || steps.restore-cache.outputs.cache-hit != 'true' }}

- name: Setup Python
uses: actions/setup-python@v3
with:
python-version: "3.8"
if: ${{ inputs.testing == 'True' || steps.restore-cache-sisl.outputs.cache-hit != 'true' }}

- name: Setup Conan and Export Recipes
- name: Export Recipes
run: |
python -m pip install --upgrade pip
python -m pip install conan~=1.0
python -m pip install gcovr
conan user
conan profile new --detect default
conan export deps/sisl/3rd_party/folly folly/2022.01.31.00@
conan export deps/sisl/3rd_party/gperftools
conan export deps/sisl/3rd_party/jemalloc
conan export deps/sisl/3rd_party/prerelease_dummy
conan export deps/sisl/3rd_party/pistache pistache/cci.20201127@
conan export 3rd_party/folly folly/2022.01.31.00@
conan export 3rd_party/gperftools
conan export 3rd_party/jemalloc
conan export 3rd_party/prerelease_dummy
conan export 3rd_party/pistache pistache/cci.20201127@
conan export 3rd_party/date date/3.0.1@
cached_pkgs=$(ls -1d ~/.conan/data/*/*/*/*/export 2>/dev/null | sed 's,.*data/,,' | cut -d'/' -f1,2 | paste -sd',' - -)
echo "::info:: Pre-cached: ${cached_pkgs}"
if: ${{ inputs.testing == 'True' || steps.restore-cache-sisl.outputs.cache-hit != 'true' }}

- name: Fixup libstdc++
run: |
# Set std::string to non-CoW C++11 version
sed -i 's,compiler.libcxx=libstdc++$,compiler.libcxx=libstdc++11,g' ~/.conan/profiles/default
if: ${{ inputs.platform == 'ubuntu-22.04' && ( inputs.testing == 'True' || steps.restore-cache-sisl.outputs.cache-hit != 'true' ) }}
if: ${{ inputs.testing == 'True' || steps.restore-cache.outputs.cache-hit != 'true' }}

- name: Build Cache
run: |
Expand All @@ -139,60 +117,45 @@ jobs:
-o malloc_impl=${{ inputs.malloc-impl }} \
-s build_type=${{ inputs.build-type }} \
--build missing \
deps/sisl
if: ${{ steps.restore-cache-sisl.outputs.cache-hit != 'true' }}
.
if: ${{ steps.restore-cache.outputs.cache-hit != 'true' }}

- name: Clean Package Cache
run: |
dep_pkgs=$(ls -1d deps/sisl/3rd_party/* 2>/dev/null | cut -d'/' -f4 | paste -sd'|' - -)
if [ -z "${dep_pkgs}" ]; then
dep_pkgs="no_3rd_party"
fi
dirty_pkgs=$(ls -1d ~/.conan/data/*/*/*/*/build 2>/dev/null | sed 's,.*data/,,')
if [ -z "${dirty_pkgs}" ]; then
dirty_pkgs="no_public/0"
fi
dirty_pkgs_d=$(echo "${dirty_pkgs}" | cut -d'/' -f1 | paste -sd'|' - -)
echo "::info:: Caching: ${dirty_pkgs_d}|${dep_pkgs}"
ls -1d ~/.conan/data/* | grep -Ev "(${dirty_pkgs_d}|${dep_pkgs})" | xargs rm -rf
rm -rf ~/.conan/data/*/*/*/*/build
rm -rf ~/.conan/data/*/*/*/*/source
if: ${{ github.event_name != 'pull_request' && steps.restore-cache-sisl.outputs.cache-hit != 'true' }}

- name: Save Sisl Cache
id: save-cache-sisl
uses: actions/cache/save@v3
- name: Save Conan Cache
uses: eBay/sisl/.github/actions/store_conan@stable/v8.x
with:
path: |
~/.conan/data
key: SislDeps8-${{ inputs.platform }}-${{ inputs.build-type }}-${{ inputs.malloc-impl }}-${{ inputs.prerelease }}-${{ hashFiles('**/conanfile.py') }}
if: ${{ github.event_name != 'pull_request' && steps.restore-cache-sisl.outputs.cache-hit != 'true' }}
key_prefix: SislDeps8-${{ inputs.platform }}-${{ inputs.build-type }}-${{ inputs.malloc-impl }}-${{ inputs.prerelease }}
if: ${{ github.event_name != 'pull_request' && steps.restore-cache.outputs.cache-hit != 'true' }}

- name: Create and test Package
- name: Code Coverage Run
run: |
if [[ "${{ inputs.build-type }}" == "Debug" && "${{ inputs.malloc-impl }}" == "libc" && "${{ inputs.prerelease }}" == "False" ]]; then
conan install \
-o prerelease=${{ inputs.prerelease }} \
-o malloc_impl=${{ inputs.malloc-impl }} \
-o coverage=True \
-s build_type=${{ inputs.build-type }} \
--build missing \
deps/sisl
conan build deps/sisl
else
sanitize=$([[ "${{ inputs.build-type }}" == "Debug" && "${{ inputs.malloc-impl }}" == "libc" && "${{ inputs.prerelease }}" == "True" ]] && echo "True" || echo "False")
conan create \
-o sisl:prerelease=${{ inputs.prerelease }} \
-o sisl:malloc_impl=${{ inputs.malloc-impl }} \
-o sisl:sanitize=${sanitize} \
-s build_type=${{ inputs.build-type }} \
--build missing \
deps/sisl
fi
if: ${{ inputs.testing == 'True' }}
conan install \
-o prerelease=${{ inputs.prerelease }} \
-o malloc_impl=${{ inputs.malloc-impl }} \
-o coverage=True \
-s build_type=${{ inputs.build-type }} \
--build missing \
.
conan build .
if: ${{ inputs.testing == 'True' && inputs.platform == 'ubuntu-22.04' && inputs.build-type == 'Debug' && inputs.malloc-impl == 'libc' && inputs.prerelease == 'False' }}

- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v3
with:
token: ${{ secrets.CODECOV_TOKEN }}
gcov: true
if: ${{ inputs.testing == 'True' && inputs.platform == 'ubuntu-22.04' && inputs.build-type == 'Debug' && inputs.malloc-impl == 'libc' && inputs.prerelease == 'False' }}

- name: Create and Test Package
run: |
sanitize=$([[ "${{ inputs.build-type }}" == "Debug" && \
"${{ inputs.malloc-impl }}" == "libc" && \
"${{ inputs.prerelease }}" == "True" ]] && \
echo "True" || echo "False")
conan create \
-o sisl:prerelease=${{ inputs.prerelease }} \
-o sisl:malloc_impl=${{ inputs.malloc-impl }} \
-o sisl:sanitize=${sanitize} \
-s build_type=${{ inputs.build-type }} \
--build missing \
.
if: ${{ inputs.testing == 'True' && ( inputs.platform != 'ubuntu-22.04' || inputs.build-type != 'Debug' || inputs.malloc-impl != 'libc' || inputs.prerelease != 'False' ) }}
13 changes: 13 additions & 0 deletions .github/workflows/merge_conan_build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,16 @@ jobs:
malloc-impl: ${{ matrix.malloc-impl }}
prerelease: ${{ matrix.prerelease }}
testing: 'True'
ChainBuild:
runs-on: "ubuntu-22.04"
steps:
- name: Start IOManager Build
run: |
curl -L \
-X POST \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.CHAIN_BUILD_TOKEN }}"\
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/eBay/iomanager/actions/workflows/merge_conan_build.yml/dispatches \
-d '{"ref":"stable/v8.x","inputs":{}}'
if: ${{ github.ref == 'refs/heads/stable/v8.x' }}
29 changes: 29 additions & 0 deletions 3rd_party/date/conandata.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
sources:
"3.0.1":
url: "https://github.com/HowardHinnant/date/archive/refs/tags/v3.0.1.tar.gz"
sha256: "7a390f200f0ccd207e8cff6757e04817c1a0aec3e327b006b7eb451c57ee3538"
"3.0.0":
url: "https://github.com/HowardHinnant/date/archive/refs/tags/v3.0.0.tar.gz"
sha256: "87bba2eaf0ebc7ec539e5e62fc317cb80671a337c1fb1b84cb9e4d42c6dbebe3"
"2.4.1":
url: "https://github.com/HowardHinnant/date/archive/refs/tags/v2.4.1.tar.gz"
sha256: "98907d243397483bd7ad889bf6c66746db0d7d2a39cc9aacc041834c40b65b98"
patches:
"3.0.1":
- patch_file: "patches/cmake-3.0.1.patch"
patch_description: "Disable string view to workaround clang 5 not having it"
patch_type: "portability"
"3.0.0":
- patch_file: "patches/cmake-3.0.0.patch"
patch_description: "Disable string view to workaround clang 5 not having it"
patch_type: "portability"
"2.4.1":
- patch_file: "patches/0001-fix-uwp.patch"
patch_description: "Fix Universal Windows Platform (UWP) unhandled exception support. See https://github.com/microsoft/vcpkg/pull/8151#issuecomment-531175393."
patch_type: "portability"
- patch_file: "patches/cmake.patch"
patch_description: "Add libcurl target for conan compatibility"
patch_type: "conan"
- patch_file: "patches/string_view.patch"
patch_description: "Disable string view to workaround clang 5 not having it"
patch_type: "portability"
Loading