Windows ARM64 source build - fails if parallel make jobs used (winlibs download race condition)

[andrjohns]

Currently experiencing this on Windows ARM64 (so not sure if present in x86) with all packages that need winlibs static libs (where the library isn't already in rtools).

If I have MAKEFLAGS=-j2 (or any value > 1) set in my .Renviron, the installation will fail at the linking stage due to libraries not being found:

install.packages("curl", repos = "https://cloud.r-project.org")
...
"C:/PROGRA~1/R-AARC~1/R-devel/bin/Rscript.exe" "../tools/winlibs.R"
clang -shared -s -static-libgcc -o curl.dll tmp.def callbacks.o curl.o download.o escape.o fetch.o findport.o form.o getdate.o handle.o ieproxy.o init.o interrupt.o multi.o nslookup.o options.o reflist.o split.o ssl.o typechecking.o utils.o version.o winidn.o writer.o -L../windows/libcurl/libclang-17.0.6 -L../windows/libcurl/lib -lwinhttp -lcurl -lnghttp2 -lssh2 -lz -lssl -lcrypto -pthread -lgdi32 -lws2_32 -lcrypt32 -lbcrypt -lwldap32 -L/aarch64-w64-mingw32.static.posix/lib -L/aarch64-w64-mingw32.static.posix/lib -LC:/PROGRA~1/R-AARC~1/R-devel/bin -lR
lld: error: unable to find library -lnghttp2
clang-17: error: linker command failed with exit code 1 (use -v to see invocation)
[1] TRUE
echo '#include <curl/curl.h>' | clang  -I../windows/libcurl/include -DCURL_STATICLIB -DSTRICT_R_HEADERS -DR_NO_REMAP -std=gnu99 -E -xc - | grep "^[ \t]*CURLOPT_.*," | sed s/,// > ../tools/option_table.txt
no DLL was created
ERROR: compilation failed for package 'curl'

Which appears to be caused by $(SHLIB) target reaching the linking stage before the winlibs target has completed. I'd guess that this hasn't been seen to-date on x86_64 because compilation is generally slower than under ARM, so the winlibs target has usually finished before $(SHLIB) reaches linking.

From local testing, an easy fix appears to be just making the dependency explicit in Makevars.win:

- all: clean winlibs
+ $(SHLIB): $(OBJECTS)
+ $(OBJECTS): winlibs

But I'm not sure whether there's an easier fix which doesn't require updating all winlibs R packages. Thanks!

[jeroen]

Yes I think that is the only solution. FWIW parallel make on Windows was never really supported because there were many other issues with it, so we didn't bother to test it.