jdk-18 build bootjdk downloads from api.adoptium.net failing

[andrew-m-leonard]

AIX build fails: https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk/job/jdk-aix-ppc64-hotspot/

00:31:13  Downloading GA release of boot JDK version 17 from https://api.adoptium.net/v3/binary/latest/17/ga/aix/ppc64/jdk/hotspot/normal/adoptium
00:31:13  
00:31:13  gzip: stdin: unexpected end of file
00:31:13  tar: Child returned status 1
00:31:13  tar: Error is not recoverable: exiting now
00:31:13  Downloading GA release of boot JDK version 17 failed.
00:31:13  Attempting to download EA release of boot JDK version 17 from https://api.adoptium.net/v3/binary/latest/17/ea/aix/ppc64/jdk/hotspot/normal/adoptium
00:31:13  
00:31:13  gzip: stdin: unexpected end of file
00:31:13  tar: Child returned status 1
00:31:13  tar: Error is not recoverable: exiting now

Other platforms fail:

10:04:41  Downloading GA release of boot JDK version 17 from https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk/job/jdk-linux-x64-hotspot/496/artifact/workspace/target/OpenJDK-jdk_x64_linux_hotspot_2021-10-05-08-04.tar.gz
10:04:42  
10:04:42  gzip: stdin: unexpected end of file
10:04:42  tar: Child returned status 1
10:04:42  tar: Error is not recoverable: exiting now
10:04:42  Downloading GA release of boot JDK version 17 failed.
10:04:42  Attempting to download EA release of boot JDK version 17 from https://api.adoptium.net/v3/binary/latest/17/ea/linux/x64/jdk/hotspot/normal/adoptium
10:04:42  
10:04:42  gzip: stdin: unexpected end of file
10:04:42  tar: Child returned status 1
10:04:42  tar: Error is not recoverable: exiting now

but then succeed when downloading from: https://api.adoptopenjdk.net

Added some debug (removed -q), and get:

11:02:42  Downloading GA release of boot JDK version 17 from https://api.adoptium.net/v3/binary/latest/17/ga/linux/x64/jdk/hotspot/normal/adoptium
11:02:42  --2021-10-05 10:02:41--  https://api.adoptium.net/v3/binary/latest/17/ga/linux/x64/jdk/hotspot/normal/adoptium
11:02:42  Resolving api.adoptium.net... 20.62.244.126
11:02:42  Connecting to api.adoptium.net|20.62.244.126|:443... connected.
11:02:42  ERROR: cannot verify api.adoptium.net's certificate, issued by `/C=US/O=Let\'s Encrypt/CN=R3':
11:02:42    Issued certificate has expired.
11:02:42  To connect to api.adoptium.net insecurely, use `--no-check-certificate'.

[andrew-m-leonard]

Seems to have started failing from 1st October

[gdams]

The SSL cert for api.adoptium.net is valid

[andrew-m-leonard]

andrewleonard@anleonar-mac Downloads % openssl s_client -servername api.adoptium.net -connect api.adoptium.net:443 | openssl x509 -noout -dates
depth=1 O = Digital Signature Trust Co., CN = DST Root CA X3
verify error:num=10:certificate has expired
notAfter=Sep 30 14:01:15 2021 GMT

[andrew-m-leonard]

https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/

[andrew-m-leonard]

Seems like: "all clients of your API must trust ISRG Root X1" @johnoliver ?

[sxa]

This issue is more pervasive than just the boot JDK downloads.

• To start to resolve the subject issue, I have put a Java17 (EA, since we haven't GA'd that version yet) in place on the AIX machines, and I have installed the CentOS7 ca-certificates package on the two Linux/s390x RHEL machines. Once deployed across all machines, this playbook change by @Haroon-Khel will put a java 17 boot JDK in place on all systems so it shouldn't have to download it during the build.
• The devtoolset-2 installation in the playbooks is failing on CentOS6 as people.centos.com is using a problematic certificate. I have submitted this PR to resolve that which removes the devtoolset-2 installation which I believe is only required for GCC 4.8 now, although it's possible the builds may need the supplementary tools like ld or others from there. The VPC run will hopefully expose that (If this is not resolved, it won't be possible to rebuild the docker image used for building so we'll be stuck)
• I have put in this PR to use curl instead of wget for the freemarker download which is currently blocking some OpenJ9 builds. At least on some systems, curl is using a more up to date CA certs bundle which is not showing the problems with wget's use of the system keystores.

[sxa]

That's not going to work :-( adoptium/infrastructure#2339 (comment)

[sxa]

Final PR of the above three has been modified to use curl to download the boot JDKs as well as for the freemarker downloads. This should resolve the failng builds problem in the short term, but we still currently cannot rebuild the dockerbuild images for LInux.x64.

[sxa]

I think this is all sorted now since the CentOS6 workaround was put in place. Please re-open if it recurs.