Some packages cannot be built 2: Electric boogaloo #21130

TomJo2000 · 2024-08-15T06:06:51Z

Important

Build validation should be done without use of the "fast build" -i/-I flags!

Lines 417 to 418 in ed5c640

 [ "$TERMUX_ON_DEVICE_BUILD" = "false" ] && echo " -i Download and extract dependencies instead of building them." 

 echo " -I Download and extract dependencies instead of building them, keep existing $TERMUX_BASE_DIR files."

e.g.

./scripts/run-docker.sh ./build-package.sh -f [package [, ...]]

We want to verify that every package can be "bootstrapped" without the need for pre-compiled dependencies.

Same deal as [Bug]: Some packages cannot be built #20966 but tested against NDK 27
Testing setup: ./scripts/run-docker.sh ./build-all.sh
- Commit: 5aaa87d (Batch 1 and 2), de39934 (Batch 3), 51516dd (Batch 4)
- Additional patches applied:
  Fixes and improvements to scripts for working with cyclic dependencies #20513
  toolchain: add API guard for timezone_t #21120 (Batch 4 onwards)
- buildorder.txt
- buildstatus.txt

List of packages that ran into build issues

Batch 1

Batch 2

Batch 3 - now at commit de39934

Batch 4 - now at commit 51516dd + #21120 applied

tvheadend (-Wunused-but-set-variable)
eltclsh (license)
jack-example-tools (Did not find CMake 'cmake')
pypy (Related to bionic-host, pypy: bump to 7.3.16 #21197)
wasn't this supposed to be fixed in pypy: fix build #21026?
python-scipy (returning, ERROR: Could not find a version that satisfies the requirement numpy==1.26.5)
gdb
gobject-introspection (Download of gobject-introspection from https://packages-cf.termux.dev/apt/termux-main failed)

Batch 5 - now at commit 9016ef0 + #21120 applied

I'll update this as I go.

The text was updated successfully, but these errors were encountered:

Fixes the current rust 1.80 build failure (#21130).

Switch from freedesktop.org to www.freedesktop.org as the former has an invalid certificate, fixing the build error noted in #21130.

Fixes the build failure in (#21130) by backporting kakoune-lsp/kakoune-lsp@bb9734c As kak-lsp does seem to have rather regular releases, another alternative could be awaiting the next release which will include this.

Fixes the current rust 1.80 build failure (#21130).

Fixes the build failure in (#21130) by backporting kakoune-lsp/kakoune-lsp@bb9734c As kak-lsp does seem to have rather regular releases, another alternative could be awaiting the next release which will include this.

Switch from freedesktop.org to www.freedesktop.org as the former has an invalid certificate, fixing the build error noted in #21130.

TomJo2000 · 2024-08-15T15:08:51Z

Started the second batch, looks like mostly simple linker problems so far.

Maxython · 2024-08-15T17:11:31Z

glibc-repo (complains about cgct, Maxy's problem)

I know the reason for this error

TomJo2000 · 2024-08-15T17:12:50Z

I know the reason for this error

Figured it'd be quicker to just leave that one for you to figure out.

TomJo2000 · 2024-08-16T04:33:57Z

Just finished the second batch, that officially puts us at 51 problems.
And 791/2517

See https://github.com/s-yata/marisa-trie/blob/master/COPYING.md Fixes build failure noted in #21130

TomJo2000 · 2024-08-17T00:56:48Z

We're up to 80 now.
I wanna get to 50%, and then I think I'm gonna call it a night.

TomJo2000 · 2024-08-17T17:59:52Z

148 now.
1557/2517, 61.86%

I've decided to clean up my build container,
rebase to the latest commit,
and apply #21120.

I think we get that everything that depends on coreutils is gonna be effected by it, so I'm just gonna save myself the hassle of writing it down another 20 times.

TomJo2000 · 2024-08-17T21:16:32Z

@Maxython I can't get gobject-introspection to build.
You might wanna look into that, seeing as its half the reason for 20513.

TomJo2000 · 2024-10-16T13:32:16Z

I'll do a recheck later today.

robertkirkman · 2024-10-17T10:02:03Z

sl (curses.h, can't reproduce)

After applying all the WIP PRs in this thread, I can now consistently reproduce that error by using scripts/run-docker.sh ./build-all.sh and waiting until sl is built, (skipping only one package currently, hilbish, which appears to have a build issue tracked in the repository of one of its dependencies)

Here's what the full context looks like for that error with sl when it's reproduced:

INFO: Identifying files with nproc=32
INFO: Done ... 0s
INFO: Found 1 / 2 files
INFO: Running symbol checks on 1 files with nproc=32
INFO: Done ... 0s
termux - build of  'skate' done
done in 14 sec
Building sl... termux - building  sl for arch aarch64...
Building dependency ncurses if necessary...
[email protected] built - skipping (rm /data/data/.built-packages/ncurses to force rebuild)
Downloading https://github.com/eyJhb/sl/archive/5.05.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  631k    0  631k    0     0   642k      0 --:--:-- --:--:-- --:--:-- 1280k
Applying patch: makefile.patch
aarch64-linux-android-clang  -fstack-protector-strong -Oz  -isystem/data/data/com.termux/files/usr/include -L/data/data/com.termux/files/usr/lib -Wl,-rpath=/data/data/com.termux/files/usr/lib -Wl,--enable-new-dtags -Wl,--as-needed -Wl,-z,relro,-z,now -o sl sl.c -lncurses
sl.c:51:10: fatal error: 'curses.h' file not found
   51 | #include <curses.h>
      |          ^~~~~~~~~~
1 error generated.
make: *** [Makefile:12: sl] Error 1
ERROR: See /home/builder/.termux-build/_buildall-aarch64/sl.out
tacokoneko@CORSAIR ~/code/termux/electric-boogaloo/termux-packages $ scripts/run-docker.sh 
Running container 'termux-package-builder' from image 'ghcr.io/termux/package-builder'...
builder@78bc2abe14fb:~/termux-packages$ ls /data/data/com.termux/files/usr/include/curses.h
ls: cannot access '/data/data/com.termux/files/usr/include/curses.h': No such file or directory
builder@78bc2abe14fb:~/tmp$ ls /data/data/com.termux/files/usr/lib/libncursesw.so.6.5 
/data/data/com.termux/files/usr/lib/libncursesw.so.6.5
builder@78bc2abe14fb:~/termux-packages$ ls output/ | grep curses
ncurses_6.5.20240831-1_aarch64.deb
ncurses-static_6.5.20240831-1_aarch64.deb
ncurses-ui-libs_6.5.20240831-1_aarch64.deb
ncurses-ui-libs-static_6.5.20240831-1_aarch64.deb
ncurses-utils_6.5.20240831-1_aarch64.deb
pomodoro-curses_2.5_aarch64.deb
builder@78bc2abe14fb:~/termux-packages$ mkdir ~/tmp
builder@78bc2abe14fb:~/termux-packages$ cd ~/tmp
builder@78bc2abe14fb:~/tmp$ ar x ~/termux-packages/output/ncurses_6.5.20240831-1_aarch64.deb 
builder@78bc2abe14fb:~/tmp$ tar xf data.tar.xz 
builder@78bc2abe14fb:~/tmp$ ls data/data/com.termux/files/usr/include/curses.h 
data/data/com.termux/files/usr/include/curses.h
builder@78bc2abe14fb:~/tmp$ ls data/data/com.termux/files/usr/lib/libncursesw.so.6.5 
data/data/com.termux/files/usr/lib/libncursesw.so.6.5
builder@78bc2abe14fb:~/tmp$

I can see that ncurses was built and its lib/libncursesw.so.6.5 file was installed, but its include/curses.h file is somehow missing from the prefix right at the moment that sl starts building. I'm quite sure that my PR doesn't cause that since I don't modify or delete any files in the include folder, so I believe this likely has the same root cause as the person who first tested sl in this issue experienced. I'll debug this as much as I can and try to find the root cause.

Root cause found:

termux-packages/packages/simulavr/build.sh

Lines 31 to 38 in 4ca64f8

 termux_step_post_make_install() { 

 mv "$TERMUX_PREFIX/share/doc/common" "$TERMUX_PREFIX/share/doc/simulavr" 

 # Headers are moved into their own subdirectory to prevent conflicts. 

 # Might cause issues when using them. 

 mv "$TERMUX_PREFIX/include" "$TERMUX_PREFIX/include-simulavr" 

 mkdir "$TERMUX_PREFIX/include" 

 mv "$TERMUX_PREFIX/include-simulavr" "$TERMUX_PREFIX/include/simulavr" 

 }

the simulavr package's termux_step_post_make_install() kind of renames the entire $TERMUX_PREFIX/include folder to $TERMUX_PREFIX/include-simulavr . I'll think about that and I guess try to write a modified simulavr package that doesn't do something like that to the $TERMUX_PREFIX.

Done in #21835

TomJo2000 · 2024-10-17T10:59:56Z

Seems like all the packages from the first batch are being built fine (according to all checkboxes marked). 👍

Still getting a build error with bionic-host locally

Downloading https://storage.googleapis.com/git-repo-downloads/repo
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 47141  100 47141    0     0   192k      0 --:--:-- --:--:-- --:--:--  192k
Downloading Repo source from https://gerrit.googlesource.com/git-repo

Traceback (most recent call last):
  File "/home/builder/.termux-build/bionic-host/src/.repo/repo/main.py", line 874, in <module>
    _Main(sys.argv[1:])
  File "/home/builder/.termux-build/bionic-host/src/.repo/repo/main.py", line 850, in _Main
    result = repo._Run(name, gopts, argv) or 0
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/builder/.termux-build/bionic-host/src/.repo/repo/main.py", line 294, in _Run
    result = run()
             ^^^^^
  File "/home/builder/.termux-build/bionic-host/src/.repo/repo/main.py", line 275, in <lambda>
    lambda: self._RunLong(name, gopts, argv, git_trace2_event_log) or 0
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/builder/.termux-build/bionic-host/src/.repo/repo/main.py", line 442, in _RunLong
    execute_command()
  File "/home/builder/.termux-build/bionic-host/src/.repo/repo/main.py", line 408, in execute_command
    execute_command_helper()
  File "/home/builder/.termux-build/bionic-host/src/.repo/repo/main.py", line 374, in execute_command_helper
    result = cmd.Execute(copts, cargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/builder/.termux-build/bionic-host/src/.repo/repo/subcmds/init.py", line 406, in Execute
    self._ConfigureUser(opt)
  File "/home/builder/.termux-build/bionic-host/src/.repo/repo/subcmds/init.py", line 217, in _ConfigureUser
    name = self._Prompt("Your Name", mp.UserName)
                                     ^^^^^^^^^^^
  File "/home/builder/.termux-build/bionic-host/src/.repo/repo/project.py", line 779, in UserName
    self._LoadUserIdentity()
  File "/home/builder/.termux-build/bionic-host/src/.repo/repo/project.py", line 792, in _LoadUserIdentity
    u = self.bare_git.var("GIT_COMMITTER_IDENT")
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/builder/.termux-build/bionic-host/src/.repo/repo/project.py", line 3835, in runner
    p.Wait()
  File "/home/builder/.termux-build/bionic-host/src/.repo/repo/git_command.py", line 556, in Wait
    self.VerifyCommand()
  File "/home/builder/.termux-build/bionic-host/src/.repo/repo/git_command.py", line 546, in VerifyCommand
    raise GitCommandError(
git_command.GitCommandError: GitCommandError: 'var GIT_COMMITTER_IDENT' on manifests failed
stderr: Committer identity unknown

*** Please tell me who you are.

Run

  git config --global user.email "[email protected]"
  git config --global user.name "Your Name"

to set your account's default identity.

Going into the container and running;
git config --global user.email "@termux"
git config --global user.name "Termux build container"
Makes it proceed to this:

Your identity is: Termux build container <@termux>
If you want to change this, please re-run 'repo init' with --config-name

Testing colorized output (for 'repo diff', 'repo status'):
  black    red      green    yellow   blue     magenta   cyan     white 
  bold     dim      ul       reverse 
Enable color display in this user account (y/N)?

Which is also annoying since it shouldn't pop up anything that requires user input during a build.
After that it does successfully fetch the repos though.

I have expressed my dislike for the Google repo utility in #21825 already,
and this just reinforces it.
But since it's what we have to use here there's no point moaning about it.

twaik · 2024-10-17T11:03:50Z

Going into the container and running; git config --global user.email "@termux" git config --global user.name "Termux build container" Makes it proceed to this:

It is already fixed in CI since it commits to termux-packages repo.

Makes it proceed to this:

AFAIK it does not pop up in non-interactive shell session (CI case).

TomJo2000 · 2024-10-17T11:24:00Z

Going into the container and running; git config --global user.email "@termux" git config --global user.name "Termux build container" Makes it proceed to this:

It is already fixed in CI since it commits to termux-packages repo.

Ah must have not fetched that on my test branch yet.

twaik · 2024-10-17T11:31:43Z

It is weird. In package_updates workflow it is done explicitly

termux-packages/.github/workflows/package_updates.yml

Lines 147 to 148 in 4ca64f8

	git config --global user.name "Termux Github Actions"
	git config --global user.email "[email protected]"

Can not find similar lines in packages workflow, possibly because it does not need to commit changes to repo.

In regular CI it is done by CI itself. Can not find proof but you can check repology-metadata repo, it has commits of Github Action worker. So I assume it handles some configuration.

I am not sure about Docker, bionic-host is built without it because it needs a lot of space and RAM.

TomJo2000 · 2024-10-17T13:58:20Z

I am not sure about Docker, bionic-host is built without it because it needs a lot of space and RAM.

Found a consistent way to get it to work in the local build container.

diff --git a/packages/bionic-host/build.sh b/packages/bionic-host/build.sh
index 7b87cca311..a3f5286e26 100644
--- a/packages/bionic-host/build.sh
+++ b/packages/bionic-host/build.sh
@@ -60,7 +60,7 @@ termux_step_get_source() {
 	cd ${TERMUX_PKG_SRCDIR}
 
 	cp -f ${TERMUX_PKG_BUILDER_DIR}/LICENSE.txt ${TERMUX_PKG_SRCDIR}/LICENSE.txt
-	
+
 	for i in libtinfo5 libncurses5 openssh-client; do
 		local URL="$(obtain_deb_url $i)"
 		wget "$URL" -O ${TERMUX_PKG_CACHEDIR}/${URL##*/}
@@ -72,10 +72,16 @@ termux_step_get_source() {
 
 	termux_download https://storage.googleapis.com/git-repo-downloads/repo ${TERMUX_PKG_CACHEDIR}/repo SKIP_CHECKSUM
 	chmod +x ${TERMUX_PKG_CACHEDIR}/repo
+	(
+	# Repo requires us to have a Git user name and email set.
+	# The GitHub workflow does this, but the local build container doesn't
+	[[ "$(git config --get user.name)" != '' ]] || git config --global user.name "Termux Github Actions"
+	[[ "$(git config --get user.email)" != '' ]] || git config --global user.email "[email protected]"
 	${TERMUX_PKG_CACHEDIR}/repo init \
 		-u https://android.googlesource.com/platform/manifest \
-		-b main -m ${TERMUX_PKG_BUILDER_DIR}/default.xml
+		-b main -m ${TERMUX_PKG_BUILDER_DIR}/default.xml <<< 'n'
 	${TERMUX_PKG_CACHEDIR}/repo sync -c -j32
+	)
 
 	sed -i '1s|.*|\#!'${TERMUX_PKG_SRCDIR}'/prebuilts/python/linux-x86/2.7.5/bin/python2|' ${TERMUX_PKG_SRCDIR}/bionic/libc/fs_config_generator.py
 	sed -i '1s|.*|\#!'${TERMUX_PKG_SRCDIR}'/prebuilts/python/linux-x86/2.7.5/bin/python2|' ${TERMUX_PKG_SRCDIR}/external/clang/clang-version-inc.py

We can just feed an n into the repo init command with a herestring.
For the Git User and Email I just checked if they're set and if not set them.
The subshell isn't necessary anymore, that's a leftover from an earlier idea.
I'll clean it up before I make a PR.

TomJo2000 · 2024-10-17T21:12:17Z

Okay so gforth doesn't like doing a 32 Bit build in the same environment it just did a 64 Bit build in (or vise versa), but otherwise works just fine.

robertkirkman · 2024-10-17T22:11:05Z

termux-am (probably builds, but did not finish up the build process)

Update: Reproduced with procyon-decompiler + build-all.sh

Update: Bisected to Fixes and improvements to scripts for working with cyclic dependencies #20513

Cloning a clean copy of termux/termux-packages and running its original build-all.sh using a custom buildorder.txt to temporarily bypass the original build-all.sh's subpackage-related errors fully completes, and does not hang.

git clone https://github.com/termux/termux-packages.git
cd termux-packages/
# while i was troubleshooting, fornwall pushed a commit to termux-am,
# so i need to temporarily do this here to retain a consistent test methodology
# see the next comment for all the results of all the same tests on termux-am
# performed on a termux-packages copy AFTER commit a7627acf5a8104ff5634330110ff9acd661c09d8
git checkout 70aa9cf1372220b6bacfc9ac51321f12b11558fa 
scripts/run-docker.sh
mkdir -p ~/.termux-build/_buildall-aarch64/
echo 'termux-am packages/termux-am' > ~/.termux-build/_buildall-aarch64/buildorder.txt 
echo 'procyon-decompiler packages/procyon-decompiler' >> ~/.termux-build/_buildall-aarch64/buildorder.txt
./build-all.sh

^ Result of above: success with no hang

Building termux-am or procyon-decompiler using scripts/run-docker.sh ./build-package.sh -f works, but building it using scripts/run-docker.sh ./build-all.sh always shows "BUILD SUCCESSFUL " then hangs there, while several java processes run indefinitely in the background of the container. This particular problem is not a crossbuild prefix pollution issue like some of the others, because the cause is definitely related to build-all.sh itself specifically, since it happens even when using a clean image of the termux-package-builder container and skipping all packages that are not termux-am.

Okay - scripts/run-docker.sh ./build-package.sh -f procyon-decompiler termux-am
Okay - scripts/run-docker.sh ./build-package.sh -f termux-am procyon-decompiler
Hang - scripts/run-docker.sh ./build-all.sh

If you want, it is possible to quickly reproduce the problem by applying Fixes and improvements to scripts for working with cyclic dependencies #20513 and then this temporary patch to use the build-all.sh entrypoint but skip to the part where termux-am fails to proceed normally.

--- a/build-all.sh
+++ b/build-all.sh
@@ -75,6 +75,11 @@ exec &>      >(tee -a "$BUILDALL_DIR"/ALL.out)
 trap 'echo ERROR: See $BUILDALL_DIR/${PKG}.out' ERR
 
 while read -r PKG PKG_DIR; do
+       # TEMPORARY TEST
+       if [ "$PKG" != "termux-am" ] && [ "$PKG" != "procyon-decompiler" ]; then
+               continue
+       fi
+
        # Check build status (grepping is a bit crude, but it works)
        if [ -e "$BUILDSTATUS_FILE" ] && grep -q "^$PKG\$" "$BUILDSTATUS_FILE"; then
                echo "Skipping $PKG"

This is what the log looks like at the bottom when it hangs.

> Task :app:lintVitalAnalyzeRelease
> Task :app:lintVitalReportRelease
> Task :app:lintVitalRelease
> Task :app:assembleRelease

BUILD SUCCESSFUL in 58s
36 actionable tasks: 36 executed

(Frozen)

> Task :Procyon.Reflection:javadocJar
> Task :Procyon.Reflection:sourcesJar
> Task :Procyon.Reflection:assemble
> Task :Procyon.Reflection:check
> Task :Procyon.Reflection:build

BUILD SUCCESSFUL in 12s
23 actionable tasks: 23 executed

(Frozen)

I will troubleshoot this and try to find any convenient way to bypass this hang, to complete another necessary step in the process of making build-all.sh complete non-interactively.

Fixes gradle hanging sometimes, see: - #21130 (comment) - gradle/gradle#14961

fornwall · 2024-10-18T10:28:15Z

@robertkirkman Regarding gradle hanging, check out: gradle/gradle#14961

As it's reported there that the problem no longer exists under gradle 8.9, I've merged a PR bumping the gradle version in termux-am: #21857. Does that fix the issue reliably when you test?

You're welcome to create additional PR:s for other packages, bumping gradle or (if it's not easy to bump gradle for certain packages) piping echo -n | into gradle input, as in:

-	$TERMUX_PKG_TMPDIR/gradle/gradle-$_GRADLE_VERSION/bin/gradle \
+	echo -n | $TERMUX_PKG_TMPDIR/gradle/gradle-$_GRADLE_VERSION/bin/gradle \

Fixes gradle hanging sometimes, see: - #21130 (comment) - gradle/gradle#14961

Fixes gradle hanging sometimes, see: - termux/termux-packages#21130 (comment) - gradle/gradle#14961

robertkirkman · 2024-10-18T12:12:29Z

@fornwall Thank you! Eventually, i detected that the problem also consistently bisected to #20513 , so I guess it would appear there is a problematic interaction between #20513 and Gradle < 8.9.

Here are my results for the same tests I did, but after your commit a7627ac instead of before.

`termux-am`

git clone https://github.com/termux/termux-packages.git
cd termux-packages/
scripts/run-docker.sh
mkdir -p ~/.termux-build/_buildall-aarch64/
echo 'termux-am packages/termux-am' > ~/.termux-build/_buildall-aarch64/buildorder.txt
./build-all.sh

✅ Success

git clone https://github.com/termux/termux-packages.git
cd termux-packages/
curl https://patch-diff.githubusercontent.com/raw/termux/termux-packages/pull/20513.diff | git apply -v
patch  --ignore-whitespace << 'EOF'
--- a/build-all.sh
+++ b/build-all.sh
@@ -75,6 +75,9 @@ exec &>       >(tee -a "$BUILDALL_DIR"/ALL.out)
 trap 'echo ERROR: See $BUILDALL_DIR/${PKG}.out' ERR
 
 while read -r PKG PKG_DIR; do
+       if [ "$PKG" != "termux-am" ]; then
+               continue
+       fi
        # Check build status (grepping is a bit crude, but it works)
        if [ -e "$BUILDSTATUS_FILE" ] && grep -q "^$PKG\$" "$BUILDSTATUS_FILE"; then
                echo "Skipping $PKG"

EOF
scripts/run-docker.sh ./build-all.sh

✅ Success

`procyon-decompiler`

git clone https://github.com/termux/termux-packages.git
cd termux-packages/
scripts/run-docker.sh
mkdir -p ~/.termux-build/_buildall-aarch64/
echo 'procyon-decompiler packages/procyon-decompiler' > ~/.termux-build/_buildall-aarch64/buildorder.txt
./build-all.sh

✅ Success

Tip

For anyone unaware of this subtle detail, when you directly use an entrypoint that has the full dependency graph of termux-packages enabled, the build will take a very long time to get to the actual Gradle part that we are currently troubleshooting because it will compile the entire dependency openjdk-17 from source. For full clarity, just to maintain consistency of testing methodology, in the test below I did not interfere with that phenomenon and I allowed the build to progress fully including openjdk-17 all the way until it reached that point where building procyon-decompiler itself hangs. However if someone else were hypothetically doing this same or a similar process, they could consider temporarily removing the line for dependency on openjdk-17 in order to skip that and use the copy of JDK that already comes with the package builder image. Also, if you are invoking the build-package.sh entrypoint instead of the build-all.sh entrypoint, you could use the argument -I for the same purpose instead.

termux-packages/packages/procyon-decompiler/build.sh

Line 9 in 18b8697

TERMUX_PKG_DEPENDS="openjdk-17"

git clone https://github.com/termux/termux-packages.git
cd termux-packages/
curl https://patch-diff.githubusercontent.com/raw/termux/termux-packages/pull/20513.diff | git apply -v
patch  --ignore-whitespace << 'EOF'
--- a/build-all.sh
+++ b/build-all.sh
@@ -75,6 +75,9 @@ exec &>       >(tee -a "$BUILDALL_DIR"/ALL.out)
 trap 'echo ERROR: See $BUILDALL_DIR/${PKG}.out' ERR
 
 while read -r PKG PKG_DIR; do
+       if [ "$PKG" != "procyon-decompiler" ]; then
+               continue
+       fi
        # Check build status (grepping is a bit crude, but it works)
        if [ -e "$BUILDSTATUS_FILE" ] && grep -q "^$PKG\$" "$BUILDSTATUS_FILE"; then
                echo "Skipping $PKG"

EOF
scripts/run-docker.sh ./build-all.sh

❌ Hanging

Based on your suggestion and these results, it looks like, to continue using #20513 in its unmodified form, a separate PR should also be opened to update the Gradle versions used by the builds of procyon-decompiler and any other packages affected by a similar situation. I will try to do that now.

`plantuml`

The issue is also reproducible with this final package dependent on the Gradle build tool because the current release of plantuml , 1.2024.7, pulls in Gradle 8.2. However, I believe it might be easy to cherrypick and use this commit in the upstream development branch of plantuml that bumps its Gradle version to 8.10, very similarly to what you did to fix the equivalent problem in termux-am. I'll most likely try that approach for fixing plantuml.

TomJo2000 · 2024-10-18T12:25:29Z

For anyone unaware of this subtle detail, when you directly use an entrypoint that has the full dependency graph of termux-packages enabled, the build will take a very long time to get to the actual Gradle part that we are currently troubleshooting because it will compile the entire dependency openjdk-17 from source. For full clarity, just to maintain consistency of testing methodology, in the test below I did not interfere with that phenomenon and I allowed the build to progress fully including openjdk-17 all the way until it reached that point where building procyon-decompiler itself hangs. However if someone else were hypothetically doing this same or a similar process, they could consider temporarily removing the line for dependency on openjdk-17 in order to skip that and use the copy of JDK that already comes with the package builder image.

Yep that is expected behavior with the testing procedure outlined above as we want to verify that all packages can be "bootstrapped" without needing to rely on precompiled dependencies.
Which wouldn't be available for forks using a different $PREFIX.

Since we've already validated the dependencies in this instance you can skip that and just build termux-am using:

./scripts/run-docker.sh ./build-package.sh -f -I -a all termux-am

Just be aware that validating "bootstrapability" is the main goal, so only use -I when the dependencies are already verified.

robertkirkman · 2024-10-18T13:09:22Z

./scripts/run-docker.sh ./build-package.sh -f -I -a all termux-am

Well, I have explained that the problem described in the last few comments is only reproducible specifically when using the PR listed as one of those for testing in this issue, #20513 plus specifically the build-all.sh script entrypoint, so it is not reproducible when using the build-package.sh script entrypoint. Therefore, in order to develop complete patches to solve the issue, it seems to me necessary to repeatedly invoke the build-all.sh script, or whichever specific part of it is triggering the problem, in order to perform checks to determine whether potential solutions have completely solved the issue yet.

For example, you can see that fornwall's recent commit a7627ac does not have an observable impact on the result of the entrypoint "./scripts/run-docker.sh ./build-package.sh -f -I -a all termux-am " . The impact has been observed exclusively on the entrypoint build-all.sh.

git clone https://github.com/termux/termux-packages.git
cd termux-packages
git checkout 70aa9cf1372220b6bacfc9ac51321f12b11558fa
./scripts/run-docker.sh ./build-package.sh -f -I -a all termux-am

git clone https://github.com/termux/termux-packages.git
cd termux-packages
./scripts/run-docker.sh ./build-package.sh -f -I -a all termux-am

The result in both of the examples above is exactly the same in every way other than that in the first case the resulting package built is version termux-am_0.8.0_all.deb and in the second case the resulting package built is termux-am_0.8.0-1_all.deb. So, you can see that the command you sent me is not an effective verification of the utility of fornwall's recent commit, so that is why I used build-all.sh in this case.

It is possible that maybe what you specifically mean is that I should also modify the build-all.sh a little bit more in order to apply the -I argument inside it manually, but I just avoided adding too many additional modifications on top of the build-all.sh for these tests in order to maintain the consistency of the testing methodology compared to the initial tests in the first comment where I described the problem.

Fixes gradle hanging under certain conditions in the two remaining packages where the issue could be reproduced. See: - termux#21130 (comment) - gradle/gradle#14961 - termux@a7627ac plantuml patch cherry picked from: - plantuml/plantuml@53a2a8d

Fixes gradle hanging under certain conditions in the two remaining packages where the issue could be reproduced. See: - termux#21130 (comment) - gradle/gradle#14961 - termux@a7627ac plantuml patch cherry picked from: - plantuml/plantuml@53a2a8d [no ci]

…10.2 Fixes gradle hanging under certain conditions in the two remaining packages where the issue could be reproduced. See: - termux#21130 (comment) - gradle/gradle#14961 - termux@a7627ac [no ci]

robertkirkman · 2024-10-19T17:11:34Z

I forgot to really mention this anywhere until now that I build it again, but 1 month ago i discovered that the aalib package will not clean compile even as a single package.

docker container stop termux-package-builder 
docker container rm termux-package-builder 
git clone https://github.com/termux/termux-packages.git
cd termux-packages/
scripts/run-docker.sh ./build-package.sh -f aalib

/home/builder/.termux-build/aalib/src/src/aacurses.c:74:20: error: incomplete definition of type 'struct _win_st'

However, fortunately it's very easy to fix using the aalib patch from its Arch Linux package.

I think what I was wondering is, since the aalib package in Termux doesn't need any Termux-specific patches, and all its patches in Arch Linux are not Arch Linux exclusive either they are just updated fixes for the legacy source code of aalib in general, would it be OK for me to make a PR that just deletes all the old aalib patches from termux-packages and replaces them with the more-recent Arch Linux patchset? (EDIT: that's not quite what I meant sorry, there are some Termux-specific patches so in reality I will add more patches rather than deleting all of them and replacing them)

In order to avoid spamming PRs I will wait until after my other open PR goes through the process before I start opening a bunch more.

TomJo2000 · 2024-10-19T17:19:43Z

I think what I was wondering is, since the aalib package in Termux doesn't need any Termux-specific patches, and all its patches in Arch Linux are not Arch Linux exclusive either they are just updated fixes for the legacy source code of aalib in general, would it be OK for me to make a PR that just deletes all the old aalib patches from termux-packages and replaces them with the more-recent Arch Linux patchset?

In order to avoid spamming PRs I will wait until after my other open PR goes through the process before I start opening a bunch more.

That sounds reasonable to me.
And don't worry about "spamming PRs", we've got something like 500 packages to validate here.
One extra PR isn't gonna break the camel's back.

fornwall mentioned this issue Aug 15, 2024

fix(main/fcp): Fix building with current rust 1.80 #21131

Merged

fornwall added a commit that referenced this issue Aug 15, 2024

bump(main/magic-wormhole-rs): 0.7.1

9c504a5

Fixes the current rust 1.80 build failure (#21130).

fornwall mentioned this issue Aug 15, 2024

bump(main/magic-wormhole-rs): 0.7.1 #21132

Merged

fornwall added a commit that referenced this issue Aug 15, 2024

bump(main/libwebrtc-audio-processing): 1.3

a6bb928

Switch from freedesktop.org to www.freedesktop.org as the former has an invalid certificate, fixing the build error noted in #21130.

fornwall mentioned this issue Aug 15, 2024

bump(main/libwebrtc-audio-processing): 1.3 #21133

Merged

fornwall mentioned this issue Aug 15, 2024

fix(main/kak-lsp): Backport patch to fix building with rust 1.80 #21134

Merged

fornwall added a commit that referenced this issue Aug 15, 2024

bump(main/magic-wormhole-rs): 0.7.1

4272a40

Fixes the current rust 1.80 build failure (#21130).

fornwall added a commit that referenced this issue Aug 15, 2024

bump(main/libwebrtc-audio-processing): 1.3

983514d

Switch from freedesktop.org to www.freedesktop.org as the former has an invalid certificate, fixing the build error noted in #21130.

fornwall added a commit that referenced this issue Aug 16, 2024

fix(main/marisa): Fix build by specifying license file

a2f86d6

See https://github.com/s-yata/marisa-trie/blob/master/COPYING.md Fixes build failure noted in #21130

fornwall mentioned this issue Aug 16, 2024

fix(main/marisa): Fix build by specifying license file #21143

Merged

fornwall added a commit that referenced this issue Aug 16, 2024

fix(main/marisa): Fix build by specifying license file

310ed2e

See https://github.com/s-yata/marisa-trie/blob/master/COPYING.md Fixes build failure noted in #21130

TomJo2000 mentioned this issue Aug 16, 2024

Auto update failing for gdal #21147

Closed

licy183 added packaging Issue related to building packages, not affecting end users directly tracker labels Aug 17, 2024

TomJo2000 added help wanted Help is wanted in order to solve the issue information Informational post WIP Work in progress, do not close the issue (PR) labels Aug 17, 2024

This was referenced Aug 17, 2024

chore(main/mariadb): remove test files #21153

Merged

toolchain: add API guard for timezone_t #21120

Merged

TomJo2000 mentioned this issue Aug 17, 2024

Fixes and improvements to scripts for working with cyclic dependencies #20513

Draft

TomJo2000 self-assigned this Aug 17, 2024

robertkirkman mentioned this issue Oct 16, 2024

[RFC] fix crossbuild prefix pollution #21835

Draft

TomJo2000 mentioned this issue Oct 17, 2024

fix(main/bionic-host): fix build issues for local builds #21850

Merged

fornwall added a commit that referenced this issue Oct 18, 2024

fix(main/termux-am): Update gradle version to 8.10.2

f024d44

Fixes gradle hanging sometimes, see: - #21130 (comment) - gradle/gradle#14961

fornwall mentioned this issue Oct 18, 2024

fix(main/termux-am): Update gradle version to 8.10.2 #21857

Merged

fornwall added a commit that referenced this issue Oct 18, 2024

fix(main/termux-am): Update gradle version to 8.10.2

a7627ac

Fixes gradle hanging sometimes, see: - #21130 (comment) - gradle/gradle#14961

termux-pacman-bot added a commit to termux-pacman/termux-packages that referenced this issue Oct 18, 2024

fix(main/termux-am): Update gradle version to 8.10.2

30935f0

Fixes gradle hanging sometimes, see: - termux/termux-packages#21130 (comment) - gradle/gradle#14961

robertkirkman mentioned this issue Oct 18, 2024

fix(main/{plantuml,procyon-decompiler}): Update gradle versions to 8.10.2 #21867

Open

This was referenced Oct 19, 2024

[Bug]: Several configure&make packages give binaries with cannot locate symbol "__extendsftf2" #8029

Open

[TRACKER]: Issues caused by libtool disrespecting -Wl,--as-needed in LDFLAGS #14629

Open

[Bug]: binary or interpreter not executable. #20336

Closed

twaik mentioned this issue Oct 19, 2024

[Bug]: Missing packages when building termux-tools packages #21176

Open

robertkirkman mentioned this issue Oct 19, 2024

fix(main/aalib): update patches to match Arch Linux #21894

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Some packages cannot be built 2: Electric boogaloo #21130

Some packages cannot be built 2: Electric boogaloo #21130

TomJo2000 commented Aug 15, 2024 •

edited

Loading

TomJo2000 commented Aug 15, 2024

Maxython commented Aug 15, 2024

TomJo2000 commented Aug 15, 2024

TomJo2000 commented Aug 16, 2024

TomJo2000 commented Aug 17, 2024

TomJo2000 commented Aug 17, 2024

TomJo2000 commented Aug 17, 2024

TomJo2000 commented Oct 16, 2024

robertkirkman commented Oct 17, 2024 •

edited

Loading

TomJo2000 commented Oct 17, 2024

twaik commented Oct 17, 2024

TomJo2000 commented Oct 17, 2024

twaik commented Oct 17, 2024

TomJo2000 commented Oct 17, 2024

TomJo2000 commented Oct 17, 2024 •

edited

Loading

robertkirkman commented Oct 17, 2024 •

edited

Loading

fornwall commented Oct 18, 2024 •

edited

Loading

robertkirkman commented Oct 18, 2024 •

edited

Loading

TomJo2000 commented Oct 18, 2024

robertkirkman commented Oct 18, 2024 •

edited

Loading

robertkirkman commented Oct 19, 2024 •

edited

Loading

TomJo2000 commented Oct 19, 2024

Some packages cannot be built 2: Electric boogaloo #21130

Some packages cannot be built 2: Electric boogaloo #21130

Comments

TomJo2000 commented Aug 15, 2024 • edited Loading

List of packages that ran into build issues

TomJo2000 commented Aug 15, 2024

Maxython commented Aug 15, 2024

TomJo2000 commented Aug 15, 2024

TomJo2000 commented Aug 16, 2024

TomJo2000 commented Aug 17, 2024

TomJo2000 commented Aug 17, 2024

TomJo2000 commented Aug 17, 2024

TomJo2000 commented Oct 16, 2024

robertkirkman commented Oct 17, 2024 • edited Loading

TomJo2000 commented Oct 17, 2024

twaik commented Oct 17, 2024

TomJo2000 commented Oct 17, 2024

twaik commented Oct 17, 2024

TomJo2000 commented Oct 17, 2024

TomJo2000 commented Oct 17, 2024 • edited Loading

robertkirkman commented Oct 17, 2024 • edited Loading

fornwall commented Oct 18, 2024 • edited Loading

robertkirkman commented Oct 18, 2024 • edited Loading

termux-am

procyon-decompiler

plantuml

TomJo2000 commented Oct 18, 2024

robertkirkman commented Oct 18, 2024 • edited Loading

robertkirkman commented Oct 19, 2024 • edited Loading

TomJo2000 commented Oct 19, 2024

TomJo2000 commented Aug 15, 2024 •

edited

Loading

robertkirkman commented Oct 17, 2024 •

edited

Loading

TomJo2000 commented Oct 17, 2024 •

edited

Loading

robertkirkman commented Oct 17, 2024 •

edited

Loading

fornwall commented Oct 18, 2024 •

edited

Loading

robertkirkman commented Oct 18, 2024 •

edited

Loading

`termux-am`

`procyon-decompiler`

`plantuml`

robertkirkman commented Oct 18, 2024 •

edited

Loading

robertkirkman commented Oct 19, 2024 •

edited

Loading