Use Bach in project with Maven structure #206

Thihup · 2021-01-26T01:19:10Z

Thihup
Jan 26, 2021

Hi!

Is there a way to compile projects using the Maven structure?
I'd like to give this project a try compiling the Piranha project.
Currently the project uses a standard Maven structure to compile the modules.

sormuras · 2021-01-26T02:23:48Z

sormuras
Jan 26, 2021
Maintainer

Ho, Thiago!

The file tree structure a project uses to organize its Java sources should(!) not matter to Bach. Bach infers the modular setup by locating module-info.java files, applying some heuristics to split main from test modules and generating appropriate javac's --module-source-path arguments.

So, yes, building a modular Java project using standard Maven structure is supported - perhaps with a little help/guidance/tweaks. The Piranha project looks like a very good candidate to verify this:

builds with Java 15+
many Java module declarations
a bunch of external modules are required (here, I expect some show-stoppers...)

Keen to try it out?

Call jshell https://github.com/sormuras/bach/raw/releases/16/build in the root directory
Upload the .bach/workspace/logbook.md file

0 replies

Thihup · 2021-01-26T02:34:02Z

Thihup
Jan 26, 2021
Author

Here you go
log.zip

0 replies

sormuras · 2021-01-26T03:03:11Z

sormuras
Jan 26, 2021
Maintainer

Thanks. Let's see what happpened.

All external modules...

 arquillian.container.spi, arquillian.core.spi,
 com.hazelcast.core,
 grizzly.framework, grizzly.http, grizzly.http.server, grizzly.http2,
 io.netty.all,
 jakarta.annotation, jakarta.cdi, jakarta.inject, jakarta.interceptor.api,
 jakarta.security.auth.message, jakarta.security.enterprise.api, jakarta.security.jacc,
 jakarta.servlet.jsp, jakarta.ws.rs,
 jersey.common,
 maven.plugin.annotations, maven.plugin.api,
 microprofile.jwt.auth,
 org.glassfish.soteria, org.glassfish.wasp,
 org.jboss.jandex,
 org.junit.jupiter.api, org.junit.jupiter.engine, org.junit.platform.launcher,
 org.omnifaces.eleos, org.omnifaces.exousia,
 shrinkwrap.api, shrinkwrap.descriptors.api.base, shrinkwrap.resolver.api.maven,
 smallrye.health,
 undertow.core,
 weld.api, weld.core.impl, weld.environment.common, weld.servlet.core, weld.spi

...must be present in the .bach/external-modules directory in order to compile all modules in one go. Did Bach download at least some of them?

Then, a few test-related modules were put into the main code space. That's because their module-info.java files are stored in src/main/java (and not in src/test/java) directories. For example: http/grizzly-test, http/impl-test, ...

So, Bach needs some help here: @ProjectInfo to the rescue!

If you create a .bach/build/module-info.java file with the following (best-guess) content:

@com.github.sormuras.bach.project.ProjectInfo(
  modules = {"cloud.piranha.api"},
  tests = {}
)
module build {
    requires com.github.sormuras.bach;
}

...at least one module should be created.

0 replies

Thihup · 2021-01-26T03:21:15Z

Thihup
Jan 26, 2021
Author

...must be present in the .bach/external-modules directory in order to compile all modules in one go. Did Bach download at least some of them?

Unfortunately, the folder .bach/external-modules doesn't exists. So I guess it couldn't find any of the modules used by the project.

Then, a few test-related modules were put into the main code space. That's because their module-info.java files are stored in src/main/java (and not in src/test/java) directories. For example: http/grizzly-test, http/impl-test, ...

Sure, we can skip them for now.

So, Bach needs some help here: @ProjectInfo to the rescue!

If you create a .bach/build/module-info.java file with the following (best-guess) content:
@com.github.sormuras.bach.project.ProjectInfo(
  modules = {"cloud.piranha.api"},
  tests = {}
)
module build {
    requires com.github.sormuras.bach;
}
...at least one module should be created.

Well, I tried your example and it didn't find any module.

Build error detected: Build failed: com.github.sormuras.bach.BuildException: No modules declared?!

But changing the modules to api/src/main/java/module-info.java it worked!

So now I'll use the ProjectInfo annotation to configure further. Looks very promising :)

0 replies

sormuras · 2021-01-26T03:33:17Z

sormuras
Jan 26, 2021
Maintainer

But changing the modules to api/src/main/java/module-info.java it worked!

Dooh! My fault. Thought, that passing module names was supported.

Here's the project-info of Bach 16. Showing how to help Bach loading external modules ... when building Bach.

bach/.bach/build/module-info.java

Lines 23 to 26 in 020995d

    
           links = { 
        
             @Link(module = "org.apiguardian.api", to = "org.apiguardian:apiguardian-api:1.1.1"), 
        
             @Link(module = "org.opentest4j", to = "org.opentest4j:opentest4j:1.2.0"), 
        
           },

0 replies

Thihup · 2021-01-26T03:47:22Z

Thihup
Jan 26, 2021
Author

Well, for one module it worked, but as soon as I specify two, the Javac starts to complain:
log.zip

Maybe the handling for multiple modules specified in the ProjectInfo annotation doesn't work? I've specified the following:

@com.github.sormuras.bach.project.ProjectInfo(
  version = "21.2.0-SNAPSHOT",
  modules = {"api/src/main/java/module-info.java", "naming/api/src/main/java/module-info.java"},
  tests = {}
)
module build {
    requires com.github.sormuras.bach;
}

(Both directories exists)

0 replies

sormuras · 2021-01-26T03:52:09Z

sormuras
Jan 26, 2021
Maintainer

No, seems like you found a bug in Bach. The --module-source-path argument looks ... off.

Can you please call the following command directly in your shell?

javac --release 16 --module cloud.piranha.api,cloud.piranha.naming.api --module-version 21.2.0-SNAPSHOT --module-source-path cloud.piranha.api=api/src/main/java --module-source-path cloud.piranha.naming.api=naming/api/src/main/java --module-path .bach/external-modules -encoding UTF-8 -d .bach/workspace/classes-main/16

0 replies

Thihup · 2021-01-26T03:54:29Z

Thihup
Jan 26, 2021
Author

That worked!

0 replies

Thihup · 2021-01-26T20:26:07Z

Thihup
Jan 26, 2021
Author

I've got to say: building with Bach is way faster than with Maven. Of course, the Maven build does a lot of extra stuff, but interactive development is way faster! (Running with java --module-path .bach/cache -m com.github.sormuras.bach build to avoid requesting each build)

I just want to know which way we should do to actually run the tests with JUnit. I'm struggling a little with that. I've added the module-info.java in the tests and tried to put the Tweak with JUnit. Is there somewhere any example?

0 replies

sormuras · 2021-01-26T20:40:52Z

sormuras
Jan 26, 2021
Maintainer

Thanks for your great feedback. Much appreciated.

[...] to avoid requesting each build

Yes, local launchers cut boot times by a lot: bach(.bat)

I just want to know which way we should do to actually run the tests with JUnit. I'm struggling a little with that. I've added the module-info.java in the tests and tried to put the Tweak with JUnit. Is there somewhere any example?

Best example is Bach itself.

https://github.com/sormuras/bach/blob/releases/16/.bach/build/module-info.java#L17-L22

    tests = {
      "com.github.sormuras.bach/test/java-module/module-info.java",
      "test.base/test/java/module-info.java",
      "test.integration/test/java/module-info.java",
    },
    requires = {"org.junit.platform.console"},

Here are 3 test modules in play: 1 in-module test module and 2 extra-module test modules. The first is patched at compile time into the main module from the main code space. The other 2 keep the modular boundaries intact and use other modules via their API.

Find basic information and more details about it here: https://sormuras.github.io/blog/2018-09-11-testing-in-the-modular-world

You may also copy and paste your current project-info annotation to this thread. Then I'll have a look and try to figure out, what's missing. A copy of the current logbook.md would help, too.

0 replies

Thihup · 2021-01-26T20:52:11Z

Thihup
Jan 26, 2021
Author

Sure. (Actually thanks for helping me ;)

log.zip

import com.github.sormuras.bach.project.ProjectInfo;
import com.github.sormuras.bach.project.ProjectInfo.Link;
import com.github.sormuras.bach.project.ProjectInfo.Tweak;

@ProjectInfo(
  version = "21.2.0-SNAPSHOT",
  modules = {
    "api/src/main/java/module-info.java", 
    "naming/api/src/main/java/module-info.java", "naming/impl/src/main/java/module-info.java", "naming/thread/src/main/java/module-info.java", 
    "policy/api/src/main/java/module-info.java", "policy/impl/src/main/java/module-info.java",
    "resource/api/src/main/java/module-info.java", "resource/impl/src/main/java/module-info.java",
    "servlet/api/src/main/java/module-info.java",
    "pages/wasp/src/main/java/module-info.java",
    "webapp/api/src/main/java/module-info.java", "webapp/impl/src/main/java/module-info.java", "webapp/annotationscan/src/main/java/module-info.java", "webapp/scinitializer/src/main/java/module-info.java", "webapp/tempdir/src/main/java/module-info.java", "webapp/webannotation/src/main/java/module-info.java", "webapp/webxml/src/main/java/module-info.java",
    "modular/modular/src/main/java/module-info.java",
    "http/api/src/main/java/module-info.java", "http/impl/src/main/java/module-info.java", "http/webapp/src/main/java/module-info.java", "http/grizzly/src/main/java/module-info.java", "http/embedded/src/main/java/module-info.java", "http/nano/src/main/java/module-info.java", "http/jdk/src/main/java/module-info.java", "http/undertow/src/main/java/module-info.java",
    "extension/servlet/src/main/java/module-info.java",
    "transaction/api/src/main/java/module-info.java", "transaction/impl/src/main/java/module-info.java",
    "cdi/weld/src/main/java/module-info.java",

    "nano/src/main/java/module-info.java",
    "embedded/src/main/java/module-info.java",
    "server/src/main/java/module-info.java",
  },
  tests = {
    "http/tests/src/main/java/module-info.java",
    // "http/grizzly-test/src/main/java/module-info.java",
    "http/impl-test/src/main/java/module-info.java"
    // "transaction/api/src/test/java/jakarta/transaction/tests/module-info.java"
  },

  // HTTP - Undertow
  requires = {"xnio.api"},
  
  links = {
    // Pages - WASP
    @Link(module = "org.glassfish.wasp", to = "org.glassfish.wasp:wasp:3.0.1"),

    @Link(module = "grizzly.framework", to = "org.glassfish.grizzly:grizzly-framework:3.0.0"),
    @Link(module = "grizzly.http", to = "org.glassfish.grizzly:grizzly-http:3.0.0"),
    @Link(module = "grizzly.http2", to = "org.glassfish.grizzly:grizzly-http2:3.0.0"),
    @Link(module = "grizzly.http.server", to = "org.glassfish.grizzly:grizzly-http-server:3.0.0"),

    
    @Link(module = "jakarta.cdi", to = "jakarta.enterprise:jakarta.enterprise.cdi-api:3.0.0"),

    @Link(module = "jakarta.interceptor.api", to = "jakarta.interceptor:jakarta.interceptor-api:2.0.0"),

    
    @Link(module = "undertow.core", to = "io.undertow:undertow-core:2.2.3.Final"),
    @Link(module = "xnio.api", to = "org.jboss.xnio:xnio-api:3.8.4.Final"),

    @Link(module = "jakarta.security.enterprise.api", to = "jakarta.security.enterprise:jakarta.security.enterprise-api:2.0.0"),
    @Link(module = "weld.api", to = "org.jboss.weld:weld-api:4.0.Final"),
    @Link(module = "weld.core.impl", to = "org.jboss.weld:weld-core-impl:4.0.0.Final"),
    @Link(module = "weld.spi", to = "org.jboss.weld:weld-spi:4.0.Final"),
    @Link(module = "weld.servlet.core", to = "org.jboss.weld.servlet:weld-servlet-core:4.0.0.Final"),
    @Link(module = "weld.environment.common", to = "org.jboss.weld.environment:weld-environment-common:4.0.0.Final"),

    @Link(module = "arquillian.container.spi", to = "org.jboss.arquillian.container:arquillian-container-spi:1.7.0.Alpha6"),
    @Link(module = "arquillian.core.spi", to = "org.jboss.arquillian.core:arquillian-core-spi:1.7.0.Alpha6"),

    @Link(module = "io.netty.all", to = "io.netty:netty-all:4.1.58.Final"),

    @Link(module = "jakarta.security.auth.message", to = "jakarta.authentication:jakarta.authentication-api:2.0.0"),
    @Link(module = "jakarta.security.jacc", to = "jakarta.authorization:jakarta.authorization-api:2.0.0"),

    @Link(module = "jersey.common", to = "org.glassfish.jersey.core:jersey-common:3.0.0"),

    @Link(module = "maven.plugin.annotations", to = "org.apache.maven.plugin-tools:maven-plugin-annotations:3.6.0"),
    @Link(module = "maven.plugin.api", to = "org.apache.maven:maven-plugin-api:3.6.3"),

    @Link(module = "microprofile.jwt.auth.api", to = "org.eclipse.microprofile.jwt:microprofile-jwt-auth-api:1.1.1"),
    @Link(module = "microprofile.jwt.auth", to = "org.omnifaces:microprofile-jwt-auth:1.0.1"),

    @Link(module = "shrinkwrap.api", to = "org.jboss.shrinkwrap:shrinkwrap-api:1.2.6"),
    @Link(module = "shrinkwrap.descriptors.api.base", to = "org.jboss.shrinkwrap.descriptors:shrinkwrap-descriptors-api-base:2.0.0"),
    @Link(module = "shrinkwrap.resolver.api.maven", to = "org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-api-maven:3.1.4"),

    @Link(module = "smallrye.health", to = "io.smallrye:smallrye-health:3.0.0-RC2")
  },

  tweaks = {
    @Tweak(tool = "jar(cloud.piranha.server)", with = {"--main-class", "cloud.piranha.server.ServerPiranha"}),
    @Tweak(tool = "junit", with = "")
  }

)
module build {
    requires com.github.sormuras.bach;
}

0 replies

Thihup · 2021-01-27T00:25:40Z

Thihup
Jan 27, 2021
Author

Unfortunately, we use Shrinkwrap, and it is not JPMS friendly. They use a proprietary ServiceLoader format, inside the META-INF/services; as result, the ModuleFinder will try to parse it as a service and it will fail, thus causing the whole process to not even start (FYI: com.github.sormuras.bach.BuildException: Build failed: java.lang.module.FindException: Unable to derive module descriptor for .bach/external-modules/shrinkwrap.resolver.api.jar).

I guess by now we won't be able to use Bach in the simplest form (with the ProjectInfo annotation). Maybe using the Builder API would work.

0 replies

sormuras · 2021-01-27T02:38:31Z

sormuras
Jan 27, 2021
Maintainer

You won the prize for compiling 30+ modules in one go with Bach. Nice feat!

Unfortunately, we use Shrinkwrap, and it is not JPMS friendly.
[...]
Unable to derive module descriptor for [...]shrinkwrap.resolver.api.jar

Does this happen already during test space compilation or later, when running tests with junit on the module path?

0 replies

Thihup · 2021-01-27T11:58:09Z

Thihup
Jan 27, 2021
Author

You won the prize for compiling 30+ modules in one go with Bach. Nice feat!

🏆

Does this happen already during test space compilation or later, when running tests with junit on the module path?

What's actually happening: The ProjectInfo and the log I've sent is before I tried to add JUnit configuration. But I tried to compile all the modules with Bach. Unfortunately, in one place we need the shrinkwrap-resolver-api. If we put that library in the external-modules, the build doesn't start up.

Is there a way to use the --patch-module without knowing the location of the Jar? If we could patch it to another module it could build and after that, I would be able to actually try the tests. I guess now I have a little understanding of what I need to configure to get the tests running (using the Bach itself as example).

0 replies

Thihup · 2021-01-27T12:16:42Z

Thihup
Jan 27, 2021
Author

Well, I could use the --patch-module, but I moved the shrinkwrap-api to the .bach directory and used the following tweak:

@Tweak(tool = "javac", with = {"--patch-module", "shrinkwrap.resolver.api.maven=.bach/shrinkwrap.resolver.api.jar"})

Now I'm getting: No module-info.java found in: []. I know that we don't have the module-info for all tests, but with this information it is not possible to detect from where it is coming from.

Edit: I've looked into the logs and it specifies from where it tried to find the test module. I fixed it.

0 replies

Thihup · 2021-01-27T12:18:46Z

Thihup
Jan 27, 2021
Author

(In the logs the cloud.piranha.api is showing as the 0-ea version, is it a bug?)

0 replies

Thihup · 2021-01-27T12:34:23Z

Thihup
Jan 27, 2021
Author

Oh no:
java.lang.LayerInstantiationException: Class loader (instance of): jdk.internal.loader.Loader @762405bf tried to define prohibited package name: java.sql

log.zip

0 replies

sormuras · 2021-01-27T14:45:45Z

sormuras
Jan 27, 2021
Maintainer

In the logs the cloud.piranha.api is showing as the 0-ea version, is it a bug?

Tried to find the reason for why 21.2.0-SNAPSHOT was replaced with 0-ea in this particular module only - but could find a suspicious tool call. I bet, it's a left-over from a failed build attempt. Can you delete the .bach/workspace directory and build the entire project again? If module cloud.piranha.api still reports 0-ea as its module version, then it's a bug.

Oh yes! 😎

tool = junit
args = --select-module cloud.piranha.transaction.api.tests --reports-dir .bach/workspace/reports/junit-test/cloud.piranha.transaction.api.tests

╷
└─ JUnit Jupiter ✔
   ├─ NotSupportedExceptionTest ✔
   │  ├─ testGetMessage2() ✔
   │  └─ testGetMessage() ✔
   ├─ TransactionRollbackExceptionTest ✔
   │  ├─ testGetMessage2() ✔
[...]

Oh no:
java.lang.LayerInstantiationException: Class loader (instance of): jdk.internal.loader.Loader @762405bf tried to define prohibited package name: java.sql

Well, that's unfortunate. Either a bug in Bach (wrong configuration of module layering) or a setup problem with the test module named cloud.piranha.transaction.nonxa.test. Guess, here's the situation where I need to step through the code with a debugger to see what's going wrong. Will fork piranaha the coming days...

0 replies

sormuras · 2021-01-27T15:01:58Z

sormuras
Jan 27, 2021
Maintainer

Unfortunately, we use Shrinkwrap, and it is not JPMS friendly. They use a proprietary ServiceLoader format, inside the META-INF/services; as result, the ModuleFinder will try to parse it as a service and it will fail, thus causing the whole process to not even start (FYI: com.github.sormuras.bach.BuildException: Build failed: java.lang.module.FindException: Unable to derive module descriptor for .bach/external-modules/shrinkwrap.resolver.api.jar).

@TomerFi had a similar issue. My reply reads:

My current solution regarding those kind of issues is to strictly separate plain old JARs from Java modules. In the sense of "one bad apple spoils the bunch" - if your project (be it a library or even only an application) requires a single non-modular JAR, I stay on the class-path. 100% modules is fun, 1% less and it doesn't feel right AND steals a lot of time.

Could be the show-stopper I mentioned above.

Best solution: get Shrinkwrap to behave on the module path.

0 replies

Thihup · 2021-01-27T15:09:06Z

Thihup
Jan 27, 2021
Author

Best solution: get Shrinkwrap to behave on the module path.

Sure, it would be the best way to solve some workarounds that we currently have in Piranha. Looks like the Shrinkwrap (and Arquillian, the parent project) has slowed down the development 😞

Best way would be forking it and release a new version, but I guess it is not so easy because of their proprietary service loader and the split packages across several jars.

For now we "solved" it by using --patch-module to avoid the split packages

0 replies

jbee · 2021-01-27T16:56:42Z

jbee
Jan 27, 2021
Collaborator Sponsor

@arjantijms I knew shrinkwrap would come back and bite :D

0 replies

arjantijms · 2021-01-28T22:48:42Z

arjantijms
Jan 28, 2021

@jbee

@arjantijms I knew shrinkwrap would come back and bite :D

😳 :P

But indeed, "Best solution: get Shrinkwrap to behave on the module path."

We should invest some time into that, do a PR upstream, and in the meantime use our own fork if needed.

0 replies

Thihup · 2021-01-29T13:09:48Z

Thihup
Jan 29, 2021
Author

I've got something working here: https://github.com/Thihup/shrinkwrap/tree/patches

0 replies

sormuras · 2021-01-30T11:19:33Z

sormuras
Jan 30, 2021
Maintainer

I like where this discussion and actions are headed to. 👍

0 replies

Thihup · 2021-01-30T12:46:43Z

Thihup
Jan 30, 2021
Author

Well... While the Shrinkwrap project itself was straightforward to modularize, we cannot say the same thing for the shrinkwrap-resolver. The main reason is that it requires the Maven project, and the Maven project has split packages 🙄.
Let's hope that Maven 4.0 can have JPMS support.

https://issues.apache.org/jira/browse/MNG-7037

0 replies

sormuras · 2021-01-30T17:04:13Z

sormuras
Jan 30, 2021
Maintainer

Yikes. Maven 5 ... if ever. There're sooo many artifacts that make up "the Maven project".

Ah, MNG-7037 looks promising. We'll see.

0 replies

Use Bach in project with Maven structure #206

Thihup Jan 26, 2021

Replies: 26 comments

sormuras Jan 26, 2021 Maintainer

Thihup Jan 26, 2021 Author

sormuras Jan 26, 2021 Maintainer

Thihup Jan 26, 2021 Author

sormuras Jan 26, 2021 Maintainer

Thihup Jan 26, 2021 Author

sormuras Jan 26, 2021 Maintainer

Thihup Jan 26, 2021 Author

Thihup Jan 26, 2021 Author

sormuras Jan 26, 2021 Maintainer

Thihup Jan 26, 2021 Author

Thihup Jan 27, 2021 Author

sormuras Jan 27, 2021 Maintainer

Thihup Jan 27, 2021 Author

Thihup Jan 27, 2021 Author

Thihup Jan 27, 2021 Author

Thihup Jan 27, 2021 Author

sormuras Jan 27, 2021 Maintainer

sormuras Jan 27, 2021 Maintainer

Thihup Jan 27, 2021 Author

jbee Jan 27, 2021 Collaborator Sponsor

arjantijms Jan 28, 2021

Thihup Jan 29, 2021 Author

sormuras Jan 30, 2021 Maintainer

Thihup Jan 30, 2021 Author

sormuras Jan 30, 2021 Maintainer

Thihup
Jan 26, 2021

sormuras
Jan 26, 2021
Maintainer

Thihup
Jan 26, 2021
Author

sormuras
Jan 26, 2021
Maintainer

Thihup
Jan 26, 2021
Author

sormuras
Jan 26, 2021
Maintainer

Thihup
Jan 26, 2021
Author

sormuras
Jan 26, 2021
Maintainer

Thihup
Jan 26, 2021
Author

Thihup
Jan 26, 2021
Author

sormuras
Jan 26, 2021
Maintainer

Thihup
Jan 26, 2021
Author

Thihup
Jan 27, 2021
Author

sormuras
Jan 27, 2021
Maintainer

Thihup
Jan 27, 2021
Author

Thihup
Jan 27, 2021
Author

Thihup
Jan 27, 2021
Author

Thihup
Jan 27, 2021
Author

sormuras
Jan 27, 2021
Maintainer

sormuras
Jan 27, 2021
Maintainer

Thihup
Jan 27, 2021
Author

jbee
Jan 27, 2021
Collaborator Sponsor

arjantijms
Jan 28, 2021

Thihup
Jan 29, 2021
Author

sormuras
Jan 30, 2021
Maintainer

Thihup
Jan 30, 2021
Author

sormuras
Jan 30, 2021
Maintainer