Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework the way to build more generic #9

Open
wants to merge 54 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
3b3c7b3
Rework the way to build more generic
navidemad Dec 19, 2022
2e7f87a
Plain progress to see easily the output
navidemad Dec 20, 2022
5a9088b
WIP
navidemad Feb 27, 2023
8bb036e
WIP
navidemad Mar 14, 2023
bb112b5
Update to support Docker modern way
navidemad Jul 25, 2024
e6ff9bd
Try make work CI
navidemad Jul 25, 2024
852f8b8
Try make work CI
navidemad Jul 25, 2024
ddbcbb2
Try 2 make work CI
navidemad Jul 25, 2024
596692d
Try 3 make work CI
navidemad Jul 25, 2024
93202ef
Try 4 make work CI
navidemad Jul 25, 2024
ca62444
Try 5 make work CI
navidemad Jul 25, 2024
7fca5ab
Try 6 make work CI
navidemad Jul 25, 2024
bc148a8
Try 7 make work CI
navidemad Jul 25, 2024
899be14
Try 8 make work CI
navidemad Jul 26, 2024
aa1a98a
Try 8 make work CI
navidemad Jul 26, 2024
83e682b
chmod 755 bin files
navidemad Jul 26, 2024
4f1c20e
fix stack
navidemad Jul 26, 2024
57ab9a8
fix stack
navidemad Jul 26, 2024
e73e8f8
WIP
navidemad Jul 26, 2024
3582e53
WIP
navidemad Jul 26, 2024
967530a
WIP
navidemad Jul 26, 2024
7e24565
WIP
navidemad Jul 26, 2024
df539f2
WIP
navidemad Jul 26, 2024
528c288
WIP
navidemad Jul 26, 2024
ce1d1c6
WIP
navidemad Jul 26, 2024
28fb032
WIP
navidemad Jul 26, 2024
f47c0cc
WIP
navidemad Jul 26, 2024
ca3506c
WIP
navidemad Jul 26, 2024
6958a2d
WIP
navidemad Jul 26, 2024
a6f5efd
WIPus
navidemad Jul 26, 2024
d525659
Add libtiff
navidemad Jul 26, 2024
37a5321
WIPus
navidemad Jul 26, 2024
2b3d35b
WIP
navidemad Jul 26, 2024
db06667
WIP
navidemad Jul 26, 2024
d3e13c4
WIP [skip ci]
navidemad Aug 1, 2024
1274c82
Edit readme
navidemad Aug 1, 2024
b0ba272
Edit readme
navidemad Aug 1, 2024
7655139
WIP
navidemad Aug 1, 2024
ca3bb2b
WIP
navidemad Aug 1, 2024
b67a037
WIP
navidemad Aug 1, 2024
2ecff16
WIP
navidemad Aug 1, 2024
7bbd990
WIP
navidemad Aug 1, 2024
1b5b19b
WIP
navidemad Aug 1, 2024
553fb26
WIP
navidemad Aug 1, 2024
e422c50
WIP
navidemad Aug 1, 2024
9d56302
WIP
navidemad Aug 1, 2024
1c07f9f
WIP
navidemad Aug 1, 2024
379c320
WIP
navidemad Aug 1, 2024
c755e60
WIP
navidemad Aug 1, 2024
fa7b9c5
WIP
navidemad Aug 1, 2024
d00aaef
WIP
navidemad Aug 1, 2024
dc26fcf
WIP
navidemad Aug 2, 2024
1043d1b
WIP
navidemad Aug 2, 2024
09599c3
WIP
navidemad Aug 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Empty file added .dockerignore
Empty file.
9 changes: 9 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
; http://editorconfig.org/

root = true

[*]
end_of_line = lf
insert_final_newline = true
charset = utf-8
trim_trailing_whitespace = true
17 changes: 17 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
version: 2
enable-beta-ecosystems: true
updates:
# Maintain dependencies for GitHub Actions
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
day: "saturday"
commit-message:
prefix: "gh"
target-branch: master
- package-ecosystem: "docker"
directory: "/"
schedule:
interval: daily
open-pull-requests-limit: 10
27 changes: 27 additions & 0 deletions .github/workflows/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: ci

on:
pull_request:
types: [opened, reopened, synchronize, ready_for_review]
push:
branches: [master]
paths-ignore: ["**.md"]

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.event.pull_request.head.ref || github.head_ref || github.ref }}
cancel-in-progress: true

jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
heroku_stack_version: [24]
steps:
- name: 🏠 Checkout source code from repository
uses: actions/checkout@v4
- name: 🧪 Build and run unit tests with Buildpack CI
uses: buildpack-ci/run-tests@v1
env:
STACK: heroku-${{ matrix.heroku_stack_version }}
BUILDPACK_URL: https://github.com/yespark/heroku-imagemagick-buildpack#rework
11 changes: 10 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,10 @@
.DS_Store
### Linux ###
*~

### macOS ###
.DS_Store

# Build artifacts
/build/*
!/build/configurations
!/build/heroku-*.tar.bz2
3 changes: 3 additions & 0 deletions .profile.d/imagemagick.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
export PATH="$PATH:$HOME/vendor/imagemagick/bin"
export PKG_CONFIG_PATH="$PKG_CONFIG_PATH:$HOME/vendor/imagemagick/lib/pkgconfig"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$HOME/vendor/imagemagick/lib"
88 changes: 61 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,55 +1,89 @@
# heroku-buildpack-imagemagick-heif
```
_____ _ _
\_ \_ __ ___ __ _ __ _ ___ /\/\ __ _ __ _(_) ___| | __
/ /\/ '_ ` _ \ / _` |/ _` |/ _ \/ \ / _` |/ _` | |/ __| |/ /
/\/ /_ | | | | | | (_| | (_| | __/ /\/\ \ (_| | (_| | | (__| <
\____/ |_| |_| |_|\__,_|\__, |\___\/ \/\__,_|\__, |_|\___|_|\_\
|___/ |___/
```

The rise in popularity and use of HEIF/HEIC(High Efficency Image Format) means your project's image processing also needs to be able to handle this format.
The current default version of imagemagick installed on heroku:22 dynos is a version 6.xx and does not support processing heic image files.
# Heroku Buildpack for ImageMagick [![Build Status](https://github.com/yespark/heroku-imagemagick-buildpack/workflows/ci/badge.svg)](https://github.com/yespark/heroku-imagemagick-buildpack/actions?query=workflow%3Aci)

This [Heroku buildpack](https://devcenter.heroku.com/articles/buildpacks) vendors a version of ImageMagick with **WEBP and HEIF support** binaries into your project.
This [Heroku buildpack](https://devcenter.heroku.com/articles/buildpacks) includes a version of ImageMagick with **HEIF/HEIC/AVIF/WebP/TIFF support**.

This one works was built for [**Heroku stack 20**](https://devcenter.heroku.com/articles/stack).
With the rise in popularity of HEIF/HEIC/AVIF/WebP/TIFF formats, your project needs to handle this format efficiently.
The default version of ImageMagick installed on Heroku:22/24 dynos (version 7.xx) does not support processing HEIC and AVIF image files.

The tar file in the [/build folder](./build) currently contains:
| Software | Version |
|---------------|-------------|
| ImageMagick | 7.1.1-35 |
| libwebp | 1.4.0 |
| libheif | 1.18.1 |

Version: ImageMagick 7.1.0-53
The binaries in this repository were built using Heroku Docker images with the [**build.sh**](./build.sh) script.

You will need to build a new binary if you want to use a newer or different version. To build a new binary see [How to Build a New Binary](#how-to-build-a-new-binary)
The resulting tar files are stored in the `/build` directory and utilized by the [compile script](./bin/compile).

# Usage

## Step 1 : Adding the buildpack
It's suggested that you use the latest version of the release buildpack.

To ensure the newer version of ImageMagick is found in the `$PATH` and installed first, make sure this buildpack is added to the top of the buildpack list (index 1).

You can set it using the heroku-cli.

From your projects "Settings" tab add this buildpack to your app in the 1st position:
For linux/ubuntu:
```bash
curl https://cli-assets.heroku.com/install-ubuntu.sh | sh
```

For mac:
```bash
https://github.com/yespark/heroku-imagemagick-buildpack
brew tap heroku/brew && brew install heroku
```

**NOTE:** \__To ensure the newer version of imagemagick is found in the $PATH and installed first make sure this buildpack is added to the top of the buildpack list or at "index 1"._
Then run:
```bash
heroku login
heroku buildpacks:set https://github.com/yespark/heroku-imagemagick-buildpack#rework … --index 1 -a heroku-imagemagick-dummy
```

## Step 2 : Clear the cache(**Not Sure if this is necessary**)
# Notes

Since the installation is cached you might want to clean it out due to config changes.
If needed, clear the cache due to configuration changes:

```bash
heroku plugins:install heroku-builds
heroku builds:cache:purge -a HEROKU_APP_NAME
heroku plugins:install @heroku-cli/heroku-builds
heroku builds:cache:purge -a your-app-name
```

# How to Build a New Binary (if you want to make somes changes)
## Workflow

The binary in this repo was built in a heroku:22 docker image running in a local dev environment.
However, there is a script called [**build.sh**](./build.sh) made to build a tar file through docker easily, it will be copied to the `build` directory. Then you should commit this changes to your git, and adjust the buildpack url previously mentionned just above.
This Dockerfile builds ImageMagick with updated libraries (libwebp, and libheif) on a Heroku stack.
Here's a breakdown of the process:

## Prerequisites
- Starts with a Heroku build image and sets up environment variables and arguments.
- Installs necessary dependencies like `pkg-config`, `ninja-build`, and `yasm`.
- Builds and installs the following libraries in order:
- `libwebp`
- `libheif` (including AOM as a dependency)
- `ImageMagick`
- For each library, it:
- Downloads the source code from GitHub
- Configures the build
- Compiles the library
- Installs it to `/usr/local`
- Cleans up unnecessary files and strips the shared libraries.
- Depending of the stack version, creates a tarball of the built binaries, libraries, and other necessary files.
- Updates the `ldconfig` cache to ensure the new shared libraries are recognized.
- Saves the ImageMagick configuration for reference.

- Docker installed and running in local dev environment. [Get Docker](https://docs.docker.com/get-docker/)
The result is a Docker image with updated versions of ImageMagick and its dependencies, with shared libraries (.so files) built and included in a tarball.

## Credits
## Contributing

- https://medium.com/@eplt/5-minutes-to-install-imagemagick-with-heic-support-on-ubuntu-18-04-digitalocean-fe2d09dcef1
- https://github.com/brandoncc/heroku-buildpack-vips
- https://github.com/steeple-dev/heroku-buildpack-imagemagick
- https://github.com/retailzipline/heroku-buildpack-imagemagick-heif
Contributions to the plugin are welcome. Report bugs and suggestions using GitHub Issues on the repository.

## License

The gem is available as open source under the terms of the [MIT License](https://github.com/yespark/heroku-imagemagick-buildpack/blob/master/LICENSE).
This buildpack is available as open source under the terms of the [MIT License](https://github.com/yespark/heroku-imagemagick-buildpack/blob/master/LICENSE).
89 changes: 64 additions & 25 deletions bin/compile
100644 → 100755
Original file line number Diff line number Diff line change
@@ -1,57 +1,96 @@
#!/usr/bin/env bash
# bin/compile <build-dir> <cache-dir>

set -e
### Configure environment

BUILD_DIR=$1
CACHE_DIR=$2
ENV_DIR=$3
set -o errexit # always exit on error
set -o pipefail # don't ignore exit codes when piping output
set -o nounset # fail on unset variables
unset GIT_DIR # Avoid GIT_DIR leak from previous build steps

BIN_DIR=$(cd "$(dirname "$0")"; pwd)
### Constants

BINARY_NAME="${STACK}.tar.bz2"

### Configure directories

BUILD_DIR=${1:-}
CACHE_DIR=${2:-}
ENV_DIR=${3:-}
BP_DIR=$(cd $(dirname ${0:-}); cd ..; pwd)

BIN_DIR=$(cd "$(dirname "$0:-")"; pwd)
ROOT_DIR=$(dirname "$BIN_DIR")

function indent() {
c='s/^/ /'
case $(uname) in
Darwin) sed -l "$c";;
*) sed -u "$c";;
esac
}

arrow() {
sed -u 's/^/-----> /'
}

ADD_PATH=""

function vendor() {
local binary="$1"
local path="$2"

if [ ! -f "$binary" ]; then
echo "Binary file '${binary}' not found for stack ${STACK}" | indent
exit 1
fi

echo "Fetching $binary" | indent
mkdir -p $path
tar -xj -f $binary -C $path

if [ -d "$path/bin" ]; then
export PATH="$path/bin:$PATH"
ADD_PATH="$path/bin"
else
ADD_PATH=""
fi

if [ -d "$path/lib/pkgconfig" ]; then
export PKG_CONFIG_PATH="$path/lib/pkgconfig:$PKG_CONFIG_PATH"
ADD_PKG_CONFIG_PATH="$path/lib/pkgconfig"
else
ADD_PKG_CONFIG_PATH=""
fi

export CPPPATH="$path/include:$CPPPATH"
export CPATH="$path/include:$CPATH"
export LIBRARY_PATH="$path/lib:$LIBRARY_PATH"
export LD_LIBRARY_PATH="$path/lib:$LD_LIBRARY_PATH"
ADD_CPPPATH="$path/include"
ADD_CPATH="$path/include"
ADD_LIBRARY_PATH="$path/lib"
ADD_LD_LIBRARY_PATH="$path/lib"
}

vendor "$ROOT_DIR/build/imagemagick.tar.bz2" "$BUILD_DIR/vendor/imagemagick"
echo "Vendoring binaries" | arrow
vendor "$ROOT_DIR/build/$BINARY_NAME" "$BUILD_DIR/vendor/imagemagick"

echo "Configuring build environment" | arrow
cat <<EOF > export
export MAGICK_HOME="$BUILD_DIR/vendor/imagemagick"
export MAGICK_CONFIGURE_PATH="/app/.magick"
export PATH="/app/bin:$PATH:\$PATH"
export LD_LIBRARY_PATH="\$LD_LIBRARY_PATH:$LD_LIBRARY_PATH"
export LIBRARY_PATH="\$LIBRARY_PATH:$LIBRARY_PATH"
export PKG_CONFIG_PATH="\$PKG_CONFIG_PATH:$PKG_CONFIG_PATH"
export CPPPATH="\$CPPPATH:$CPPPATH"
export CPATH="\$CPATH:$CPATH"
export PATH="/app/bin:$ADD_PATH:\$PATH"
export LD_LIBRARY_PATH="\$LD_LIBRARY_PATH:$ADD_LD_LIBRARY_PATH"
export LIBRARY_PATH="\$LIBRARY_PATH:$ADD_LIBRARY_PATH"
export PKG_CONFIG_PATH="\$PKG_CONFIG_PATH:$ADD_PKG_CONFIG_PATH"
export CPPPATH="\$CPPPATH:$ADD_CPPPATH"
export CPATH="\$CPATH:$ADD_CPATH"
EOF

echo "Building runtime environment" | arrow
mkdir -p $BUILD_DIR/.profile.d

cat <<EOF > $BUILD_DIR/.profile.d/imagemagick.sh
export MAGICK_HOME="$BUILD_DIR/vendor/imagemagick"
export MAGICK_CONFIGURE_PATH="/app/.magick"
export PATH="/app/bin:${PATH//$BUILD_DIR//app}:\$PATH"
export LD_LIBRARY_PATH="\$LD_LIBRARY_PATH:${LD_LIBRARY_PATH//$BUILD_DIR//app}"
export LIBRARY_PATH="\$LIBRARY_PATH:${LIBRARY_PATH//$BUILD_DIR//app}"
export PKG_CONFIG_PATH="\$PKG_CONFIG_PATH:${PKG_CONFIG_PATH//$BUILD_DIR//app}"
export CPPPATH="\$CPPPATH:${CPPPATH//$BUILD_DIR//app}"
export CPATH="\$CPATH:${CPATH//$BUILD_DIR//app}"
export PATH="/app/bin:${ADD_PATH//$BUILD_DIR//app}:\$PATH"
export LD_LIBRARY_PATH="\$LD_LIBRARY_PATH:${ADD_LD_LIBRARY_PATH//$BUILD_DIR//app}"
export LIBRARY_PATH="\$LIBRARY_PATH:${ADD_LIBRARY_PATH//$BUILD_DIR//app}"
export PKG_CONFIG_PATH="\$PKG_CONFIG_PATH:${ADD_PKG_CONFIG_PATH//$BUILD_DIR//app}"
export CPPPATH="\$CPPPATH:${ADD_CPPPATH//$BUILD_DIR//app}"
export CPATH="\$CPATH:${ADD_CPATH//$BUILD_DIR//app}"
EOF
12 changes: 11 additions & 1 deletion bin/detect
100644 → 100755
Original file line number Diff line number Diff line change
@@ -1,2 +1,12 @@
#!/usr/bin/env bash
echo "ImageMagick" && exit 0

SUPPORTED_STACKS=(heroku-22 heroku-24)

for stack_option in "${SUPPORTED_STACKS[@]}"; do
if [[ $STACK = $stack_option ]]; then
echo "imagemagick ($stack_option stack)"
exit 0
fi
done

echo "imagemagick buildpack doesn't support the $STACK stack" && exit 1
Empty file modified bin/release
100644 → 100755
Empty file.
31 changes: 31 additions & 0 deletions bin/test
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#!/usr/bin/env bash

cd "${1-}"

function indent() {
c='s/^/ /'
case $(uname) in
Darwin) sed -l "$c";;
*) sed -u "$c";;
esac
}

ls -lah /app/vendor/imagemagick/lib | indent

magick -list configure | indent
# magick -resize 100 fixtures/test.avif fixtures/test-resize.avif | indent
# magick -resize 100 fixtures/test.heic fixtures/test-resize.heic | indent
# magick -resize 100 fixtures/test.jpg fixtures/test-resize.jpg | indent
# magick -resize 100 fixtures/test.jpg webp:fixtures/test-jpg.webp | indent
# magick -resize 100 fixtures/test.jpg avif:fixtures/test-jpg.avif | indent
# magick -resize 100 fixtures/test.png fixtures/test-resize.png | indent
# magick -resize 100 fixtures/test.png webp:fixtures/test-png.webp | indent
# magick -resize 100 fixtures/test.png avif:fixtures/test-png.avif | indent
# magick -resize 100 fixtures/test.png jxl:fixtures/test-png.jxl | indent
# magick -resize 100 fixtures/test.tiff fixtures/test-resize.tiff | indent
# magick -resize 100 fixtures/test.tiff webp:fixtures/test-tiff.webp | indent
# magick -resize 100 fixtures/test.tiff avif:fixtures/test-tiff.avif | indent
# magick -resize 100 fixtures/test.tiff jxl:fixtures/test-tiff.jxl | indent
# magick fixtures/jpg-with-metadata.jpg fixtures/jpg-with-metadata.webp | indent
# magick -coalesce fixtures/test.gif fixtures/test-gif.webp | indent
# magick fixtures/test.jpg -antialias -font DejaVu-Sans -pointsize 20 -gravity Southeast -annotate +15+15 'TEST' fixtures/test-text.jpg | indent
4 changes: 4 additions & 0 deletions bin/test-compile
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/bash
# usage: bin/test-compile <build-dir> <cache-dir> <env-dir>

"$(dirname ${0:-})/compile" "$1" "$2" "$3"
Loading