Skip to content

Commit

Permalink
[major] Add GitHub Actions configuration (#523)
Browse files Browse the repository at this point in the history
- build and publish both Fedora 28 RPM and image
- adjust build scripts for GitHub Actions
  - s/CIRCLE_BRANCH/GITHUB_REF_NAME/
  - s/CIRCLE_BUILD_NUM/GITHUB_RUN_NUMBER/
  - s/CIRCLE_SHA1/GITHUB_SHA/
- rename RPM to simply `fusedav` (was `fusedav-release`,
  `fusedav-dev`, `fusedav-stage`, or `fusedav-yolo`)
- refactor/relocate computation of fusedav version string
- also: add GitHub Actions plugin and mount .netrc within dev
  container
  • Loading branch information
djschaap committed Nov 16, 2023
1 parent 0db522c commit f9a793c
Show file tree
Hide file tree
Showing 11 changed files with 459 additions and 169 deletions.
7 changes: 7 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
"settings": {},
"extensions": [
"davidanson.vscode-markdownlint",
"github.vscode-github-actions",
"ms-azuretools.vscode-docker",
"ms-vscode.cpptools-extension-pack",
"ms-vscode.makefile-tools",
Expand All @@ -20,13 +21,19 @@
"settings": {},
"extensions": [
"davidanson.vscode-markdownlint",
"github.vscode-github-actions",
"ms-azuretools.vscode-docker",
"ms-vscode.cpptools-extension-pack",
"ms-vscode.makefile-tools",
"redhat.vscode-yaml"
]
}
},
// Bind mounting ~/.netrc allows use of the GitHub CLI (`gh`) by running
// `export GH_TOKEN=$(awk '{print $6}' ~/.netrc)` within the dev container.
"mounts": [
"type=bind,source=${localEnv:HOME}/.netrc,target=/home/vscode/.netrc,readonly"
],
"containerUser": "vscode",
"updateRemoteUserUID": true
}
1 change: 0 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
.git/
.gitignore
.vscode/
LATEST_RPM
README.md
VERSION
cache/
Expand Down
95 changes: 95 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
name: Build everything
on:
- push
env:
REGISTRY: ghcr.io
defaults:
run:
shell: bash
# specifying `bash` here ensures `set -eo pipefail` is active
jobs:
build-everything:
runs-on: ubuntu-latest
permissions:
# contents:write allows creating a GitHub Release.
# packages:write allows publishing an image to GitHub Packages.
contents: write
packages: write
steps:
- name: Check out repository code
uses: actions/checkout@v4
with:
# autotag requires a reasonably complete git history.
fetch-depth: 0
- name: Build fusedav-dev image
run: docker build --progress plain --target dev -t fusedav-dev .
- name: Install autotag
run: |
curl -fsSL https://github.com/pantheon-systems/autotag/releases/latest/download/autotag_linux_amd64 \
-o /usr/local/bin/autotag
chmod 0755 /usr/local/bin/autotag
- name: Generate new version strings and tag(s)
env:
IMAGE_NAME: ${{ github.repository }}
run: |
echo "new-version.sh:"
scripts/compute-version.sh | tee new-version.sh
- name: Build/tag fusedav image
run: |
echo START build target extract
docker build --progress plain --target extract . --output extract
echo DONE build target extract
echo
echo START build final image
. new-version.sh
# Use older "maintainer" label instead of "org.opencontainers.image.maintainer"
# to overwrite the docker.io/library/fedora:28 value.
docker build --progress plain --target compile \
--label "[email protected]" \
--label "org.opencontainers.image.description=FUSE-based DAV client with extensions for performance" \
--label "org.opencontainers.image.licenses=GPLv2" \
--label "org.opencontainers.image.source=https://github.com/pantheon-systems/fusedav" \
--label "org.opencontainers.image.vendor=Pantheon Systems, Inc." \
--label "org.opencontainers.image.version=${SEMVER}" \
-t fusedav .
echo DONE build final image
for tag in ${IMAGE_TAGS[@]}; do
echo "tag image as ${tag}"
docker tag fusedav $tag
done
- name: Log in to GitHub Container Registry
uses: docker/login-action@65b78e6e13532edd9afa3aa52ac7964289d1a9c1
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Push final image/tag(s) to container registry
run: |
. new-version.sh
for tag in ${IMAGE_TAGS[@]}; do
echo "push ${tag}"
docker push $tag
done
- name: Create (pre-release) GitHub release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
. new-version.sh
# If NOT master, create a pre-release.
# Note that GitHub converts the tilde (`~`) which indicates a pre-release
# RPM version to a period (`.`) in the filename. This should not affect
# RPM version comparison operations.
echo "Create pre-release release:"
gh release create $GITHUB_RELEASE_NAME -p --generate-notes
. scripts/upload-gh-assets.sh
if: github.ref != 'refs/heads/master'
- name: Create GitHub release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
. new-version.sh
# Iff master, create a regular release.
echo "Create regular release:"
gh release create $GITHUB_RELEASE_NAME --generate-notes
. scripts/upload-gh-assets.sh
if: github.ref == 'refs/heads/master'
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@
/tests/readwhatwaswritten
/tests/rename
/tests/trunc
LATEST_RPM
Makefile.in
VERSION
_trial_temp*
Expand Down
55 changes: 19 additions & 36 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,6 @@
#
# runtime is the final runtime image.
#
# To build image locally:
# docker build --progress plain --build-arg CIRCLE_SHA1=$(git rev-parse --short HEAD) -t fusedav .
#
# To compile/build and extract the RPM into the `extract` directory:
# docker build --progress plain --build-arg GITHUB_SHA=$(git rev-parse --short HEAD) --target extract -t fusedav-extract . --output=extract
#
FROM docker.io/library/fedora:28 AS base

SHELL ["/bin/bash", "-euo", "pipefail", "-c"]
Expand Down Expand Up @@ -50,6 +44,7 @@ FROM base AS dev

RUN \
dnf install -y \
'dnf-command(config-manager)' \
autoconf \
automake \
bind-utils \
Expand All @@ -71,6 +66,8 @@ RUN \
tcpdump \
uriparser-devel \
zlib-devel \
&& dnf config-manager --add-repo https://cli.github.com/packages/rpm/gh-cli.repo \
&& dnf install -y gh \
&& dnf clean all \
&& rm -rf /var/cache/dnf \
&& curl -fsSL https://github.com/pantheon-systems/autotag/releases/latest/download/autotag_linux_amd64 \
Expand All @@ -80,34 +77,19 @@ RUN \
# Installing autotag above makes it available within a dev container.
# When building via CI/CD, autotag is installed/called elsewhere.

# Installing gh above makes it available within a dev container.
# When building via GitHub Actions, gh is installed/called elsewhere.

USER vscode

########################################

FROM dev AS compile

# new-version.sh MUST be created before we get here
COPY . /build
WORKDIR /build

ARG CIRCLE_BRANCH="unknown"
ARG CIRCLE_BUILD_NUM=""
ARG CIRCLE_SHA1=0000000

# CHANNEL is always `release` now.
# Historically, CHANNEL could be: dev, stage, yolo, release
ARG CHANNEL=release

# Set PACKAGECLOUD_REPO to `internal` or `internal-staging` to publish RPM.
ARG PACKAGECLOUD_REPO=""

# RPM_VERSION is set here for local/direct `docker build` use; CircleCI builds
# will set their own value.
ARG RPM_VERSION="0.0.0+0"

# SEMVER is set here for local/direct `docker build` use; CircleCI builds
# will set their own value.
ARG SEMVER="0.0.0-local"

# Using explicit USER instructions instead of sudo to satisfy Guardrails.
USER root

Expand All @@ -117,27 +99,28 @@ RUN \
USER vscode

RUN \
echo "${RPM_VERSION}" > VERSION \
&& scripts/build-rpm.sh "${CHANNEL}" \
&& if [ -n "${PACKAGECLOUD_REPO}" ] ; then \
echo SKIPPING scripts/push_packagecloud.sh ; \
else \
echo "NOT pushing RPM to Packagecloud as this is a pre-release build" ; \
fi
scripts/build-rpm.sh

########################################

FROM scratch AS extract

COPY --from=compile /build/pkg pkg
COPY --from=compile /home/vscode/rpmbuild/RPMS RPMS
COPY --from=compile /home/vscode/rpmbuild/SRPMS SRPMS
COPY --from=compile /build/LATEST-RPM-VER-REL LATEST-RPM-VER-REL

########################################

FROM base AS runtime

ARG CHANNEL=release
COPY --from=compile \
/build/LATEST-RPM-VER-REL \
/home/vscode/rpmbuild/RPMS/x86_64/fusedav-*.rpm \
/tmp/

COPY --from=compile /build/src/fusedav "/opt/pantheon/fusedav-${CHANNEL}/fusedav-${CHANNEL}"
COPY scripts/exec_wrapper/mount.fusedav_chan "/usr/sbin/mount.fusedav-${CHANNEL}"
# BEWARE: `.fc28` is the RPM release suffix normally added by rpmbuild.
RUN \
LATEST=$(cat /tmp/LATEST-RPM-VER-REL) \
&& rpm -i "/tmp/fusedav-${LATEST}.fc28.x86_64.rpm"

USER fusedav
118 changes: 92 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,52 +1,118 @@
Fusedav
=======
# Fusedav

[![Unsupported](https://img.shields.io/badge/Pantheon-Unsupported-yellow?logo=pantheon&color=FFDC28)](https://pantheon.io/docs/oss-support-levels#unsupported)

Fusedav Client
--------------
## Fusedav Client

This is a fuse-based DAV client with extensions for performance.
This will run against any standard DAV implementation, i.e.
`pywebdav` (see tests).

Installation
------------
## Production (CI/CD) Build

1. ```git clone git://github.com/pantheon-systems/fusedav.git```
2. ```git clean -f -x -d && ./autogen.sh && ./configure && make```
Merging anything to the `master` branch will trigger GitHub Actions to build
a production release.

Usage
-----
The RPM(s) are published as a [Release](https://github.com/pantheon-systems/fusedav/releases).

Use the ```-V``` flag to see libraries.
Container image(s) are published as [Packages](https://github.com/pantheon-systems/fusedav/pkgs/container/fusedav).
Images are generally accessed as `ghcr.io/pantheon-systems/fusedav:0.0.0`, where
`0.0.0` is the desired version.

### RPM Retrieval / Installation

Note that, while the RPM version is removed from the filename entirely, the
RPM release `.fc28` suffix is retained. This is being done to allow for use
of other base images/distributions in the future.

```sh
# Install latest release
curl -fsSL https://github.com/pantheon-systems/fusedav/releases/latest/download/fusedav.fc28.x86_64.rpm \
-o fusedav.x86_64.rpm \
&& dnf install -y fusedav.x86_64.rpm

# Install specific release
RELEASE_NAME=v0.0.0-branch.1
curl -fsSL "https://github.com/pantheon-systems/fusedav/releases/download/${RELEASE_NAME}/fusedav.fc28.x86_64.rpm" \
-o fusedav.x86_64.rpm \
&& dnf install -y fusedav.x86_64.rpm

# Install specific release, alternate method
RELEASE_NAME=v0.0.0-branch.1
curl -fsSL "https://api.github.com/repos/pantheon-systems/fusedav/releases/tags/${RELEASE_NAME}" \
| jq ".assets[] | select(.name==\"fusedav.fc28.x86_64.rpm\") | .browser_download_url" \
| xargs curl -o fusedav.x86_64.rpm -fsSL \
&& dnf install -y fusedav.x86_64.rpm
```

## Development Build

### Development Image Build

Building a local container image requires only:

1) a local Docker (or Podman) installation, and
1) [autotag](https://github.com/pantheon-systems/autotag) in the PATH.
PATH.

```sh
scripts/compute-version.sh | tee new-version.sh
docker build --progress plain -t fusedav .
```

### Development RPM Build

Building a set of RPMs within a containerized environment requires only:

1) a local Docker (or Podman) installation, and
1) [autotag](https://github.com/pantheon-systems/autotag) in the PATH.

The generated RPMs will be written to the `extract` directory.

```sh
scripts/compute-version.sh | tee new-version.sh
docker build --progress plain --target extract . --output=extract
```

### Development Code Build

1. Clone the git repository.
- You may have done this already.
- `git clone git://github.com/pantheon-systems/fusedav.git`
1. Install build dependencies.
- See `BuildRequires` in [fusedav-template.spec](fusedav-template.spec)
for required Fedora Linux 28 packages.
- You may have done this already.
- ALTERNATIVELY, open this git repository in Visual Studio Code with
the [ms-vscode-remote.remote-containers](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers)
extension active and work within the `fedora28` (default) dev
container.
1. Compile the code.
- `git clean -f -x -d && ./autogen.sh && ./configure && make`
- The executable will be written to `src/fusedav`.

## Usage

Use the `-V` flag to see libraries.

```text
$ src/fusedav -V
fusedav version 2.0.42-bccf93b
LevelDB version 1.20
libcurl/7.59.0 OpenSSL/1.1.0i zlib/1.2.11 libidn2/2.0.5 libpsl/0.20.2 (+libidn2/2.0.4) libssh/0.8.5/openssl/zlib nghttp2/1.32.1
FUSE library version: 2.9.7
```

Debug/Develop
-----
Running this docker script in debug mode will build a fedora-22 container with the local source mounted inside it suitable to build fusedav.
```
BUILD_VERSIONS=22 BUILD_DEBUG=1 ./scripts/docker-outer.sh
```

libcurl and OpenSSL
-------------------
## libcurl and OpenSSL

FuseDAV requires libcurl linked with OpenSSL. On Fedora versions before 27 the
provided libcurl is linked against NSS and you need to provide your own libcurl
linked against OpenSSL.

Contributing
------------
## Contributing

1. Fork it.
2. Create a branch (`git checkout -b my_new_features`)
3. Commit your changes (`git commit -am "Adding a nice new feature"`)
4. Push to the branch (`git push origin my_new_feature`)
5. Open a Pull Request with relevant information
2. Create a branch (`git checkout -b my_new_features`).
3. Commit your changes (`git commit -am "Adding a nice new feature"`).
4. Push to the branch (`git push origin my_new_feature`).
5. Open a Pull Request with relevant information.
Loading

0 comments on commit f9a793c

Please sign in to comment.