Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Bzlmod and add rules_scala to bazel-central-registry #1482

Open
sfc-gh-pbennes opened this issue Mar 16, 2023 · 21 comments · Fixed by #1619 · May be fixed by #1541 or #1595
Open

Support Bzlmod and add rules_scala to bazel-central-registry #1482

sfc-gh-pbennes opened this issue Mar 16, 2023 · 21 comments · Fixed by #1619 · May be fixed by #1541 or #1595

Comments

@sfc-gh-pbennes
Copy link

sfc-gh-pbennes commented Mar 16, 2023

Greetings!

It looks like rules_scala isn't part of the Bzlmod effort or added to bazel-central-registry yet. I've opened an issue both here and in https://github.com/bazelbuild/bazel-central-registry to request it: bazelbuild/bazel-central-registry#522

@chrislovecnm
Copy link

Just a note to anyone that starts the work native.register_toolchains is not supported when using bzlmod.

@sluongng
Copy link

@chrislovecnm
Copy link

chrislovecnm commented Jul 11, 2023

@sluongng
Copy link

Yeah you would only want to prepare the toolchain (as a new repository) inside the extension I think. Registering it must be called from the MODULE.bazel file.

So native.register_toolchains call from the extension's starlark does not work, but having the extension create @pythons_hub//:all and then MODULE.bazel calling register_toolchains("@pythons_hub//:all") worked. 👍

@mateuszkuta256
Copy link
Contributor

I prepared a minimal bzlmod support here and gonna split it into a few PRs
TODOs after merging it:

  • write documentation on how to use rules_scala with bzlmod (some properties like SCALA_VERSION will probably be configurable via repo-env)
  • prepare release notes
  • contribute repo to the Bazel Central Registry
  • add tests that the project builds using both WORKSPACE and MODULE.bazel
  • migrate test/external workspaces to blzmod too

@pcj
Copy link
Member

pcj commented Jun 30, 2024

@mateuszkuta256 thanks for getting this started! Any progress updates or blockers?

@mateuszkuta256
Copy link
Contributor

@mateuszkuta256 thanks for getting this started! Any progress updates or blockers?

hi, unfortunately I'm working on other things now and won't continue with this PR in the foreseeable future
I remember this migration was on hold because interfered with 'cross compilation support'
It looks like great progress was done it this area, so maybe someone can already take over the PR

@agluszak agluszak linked a pull request Aug 7, 2024 that will close this issue
@mbland
Copy link
Contributor

mbland commented Oct 2, 2024

I'd like to take on the task of Bzlmodifying this repo through a series of pull requests.

I've already created a (mostly) working branch in my own fork. Though I saw a couple draft pull requests here, I ended up taking a different approach and got it mostly working. In fact, I did exactly what @sluongng suggested in #1482 (comment). (I didn't notice this comment before just now—I might've read it, but not understood it at the time—but I did study rules_python and rules_go, and ended up doing exactly that.)

There are still issues I need help to address (recorded in some of the commit messages), I didn't strictly maintain WORKSPACE compatibility (fixable), and I only tested against Scala 2.13. But I can start teasing chunks out of it in a methodical fashion, to ensure that I maintain WORKSPACE and cross-version compatibility.

For an example of what it looks like from a client perspective, here's what the MODULE.bazel stanza from my local EngFlow/example repo looks like (as opposed to the current stanza, explanatory comment notwithstanding):

bazel_dep(name = "rules_scala", repo_name = "io_bazel_rules_scala")
local_path_override(
    module_name = "rules_scala",
    path = "../../bazelbuild/rules_scala"
)   

scala_dev_deps = use_extension(
    "@io_bazel_rules_scala//scala/extensions:deps.bzl",
    "scala_deps",
    dev_dependency = True,
)   
scala_dev_deps.toolchains(
    scalatest = True,
)

So if folks are game for me to do this, I'll start carving off pieces as separate pull requests, and we can resolve any outstanding problems in the process.

cc: @BillyAutrey @jayconrod @benjaminp @TheGrizzlyDev

mbland added a commit to mbland/rules_scala that referenced this issue Oct 3, 2024
This begins the Bzlmod compatibility migration by updating Bazel to
version 7.3.2 and adding initial `MODULE.bazel` and `WORKSPACE.bzlmod`
files.

Part of: bazelbuild#1482

Though Bzlmod remains disabled, updating to Bazel 7.3.2 requred updating
or adding the following packages to maintain `WORKSPACE` compatibility.

In `rules_scala_setup()`:

- bazel_skylib: 1.4.1 => 1.7.1
- rules_cc: 0.0.6 => 0.0.10
- rules_java: 5.4.1 => 7.9.0
- rules_proto: 5.3.0-21.7 => 6.0.2

Dev dependencies in `WORKSPACE`:

- com_google_protobuf: 28.2
- rules_pkg: 1.0.1
- rules_jvm_external: 6.4
- com_google_absl: abseil-cpp-20240722.0
- zlib: 1.3.1

Of all of the new, explicit dev dependencies, only `com_google_protobuf`
will be necessary to include in `MODULE.bazel`. The Bzlmod mechanism
will discover these other transitive dev dependencies automatically.

Also removed the `rules_java_extra` repo from `WORKSPACE`, which
appeared unused.

---

Though the current `rules_java` version is 7.12.1, and largely works
with this repo, it requires a few temporary workarounds. Rather than
commit the workarounds, upgrading only to 7.9.0 now seems less crufty.

What follows is a very detailed explanation of what happens with 7.12.1
with Bazel 7.3.2, just to have it on the record.

---

The workaround is to change a few toolchain and macro file targets from
`@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. This isn't a
terribly bad or invasive workaround, but `@bazel_tools//tools/jdk:` is
clearly the canonical path. Best to keep it that way, lest we build up
technical debt.

Without the workaround, these targets would fail:

- //test/src/main/resources/java_sources:CompiledWithJava11
- //test/src/main/resources/java_sources:CompiledWithJava8
- //test/toolchains:java21_toolchain
- //test:JunitRuntimePlatform
- //test:JunitRuntimePlatform_test_runner
- //test:scala_binary_jdk_11

with this error:

```txt
ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14:

While resolving toolchains for target
@@rules_java_builtin//toolchains:platformclasspath (096dcc8):

No matching toolchains found for types
@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type.
```

This appears to be a consequence of both upgrading the Bazel version
from 6.3.0 to 7.3.2 and updating `rules_java` to 7.12.1. The
`rules_java_builtin` repo is part of the `WORKSPACE` prefix that adds
implicit dependencies:

- https://bazel.build/external/migration#builtin-default-deps

This repo was added to 7.0.0-pre.20231011.2 in the following change,
mapped to `@rules_java` within the scope of the `@bazel_tools` repo:

- bazelbuild/bazel: Add rules_java_builtin to the users of Java modules
  bazelbuild/bazel@ff1abb2

This change tried to ensure `rules_java` remained compatible with
earlier Bazel versions. However, it changed all instances of
`@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` to
`//toolchains:bootstrap_runtime_toolchain_type`:

- bazelbuild/rules_java: Make rules_java backwards compatible with Bazel
  6.3.0
  bazelbuild/rules_java@30ecf3f

Bazel has bumped `rules_java` in its `workspace_deps.bzl` from 7.9.0 to
7.11.0, but it's only available as of 8.0.0-pre.20240911.1.

- bazelbuild/bazel: Update rules_java 7.11.1 / java_tools 13.8
  bazelbuild/bazel#23571
  bazelbuild/bazel@f92124a

---

What I believe is happening is, under Bazel 7.3.2 and `rules_java`
7.12.1:

- Bazel creates `rules_java` 7.9.0 as `@rules_java_builtin` in the
  `WORKSPACE` prefix.

- `@bazel_tools` has `@rules_java` mapped to `@rules_java_builtin` when
  initialized during the `WORKSPACE` prefix, during which
  `@bazel_tools//tools/jdk` registers `alias()` targets to
  `@rules_java` toolchain targets. These aliased toolchains specify
  `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` in their
  `toolchains` attribute.

- `WORKSPACE` loads `@rules_java` 7.12.1 and registers all its
  toolchains with type
  `@rules_java//toolchains:bootstrap_runtime_toolchain_type`.

- Some `@rules_java` rules explicitly specifying toolchains from
  `@bazel_tools//tools/jdk` can't find them, because the
  `@bazel_tools//tools/jdk` toolchain aliases expect toolchains of type
  `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`.

This has broken other projects in the same way:

- bazelbuild/bazel: [Bazel CI] Downstream project broken by rules_java
  upgrade #23619
  bazelbuild/bazel#23619

These problems don't appear under Bzlmod, and `@rules_java_builtin` was
never required. This is because `WORKSPACE` executes its statements
sequentially, while Bzlmod builds the module dependency graph _before_
instantiating repositories (within module extensions).

It seems a fix is on the way that removes `@rules_java_builtin` from the
`WORKSPACE` prefix, and adds `@rules_java` to the suffix. At this
moment, though, it's not even in a prerelase:

- bazelbuild/bazel: Remove rules_java_builtin in WORKSPACE prefix
  bazelbuild/bazel@7506690

---

Note that the error message matches that from the following resolved
issue, but that issue was for non-Bzlmod child repos when `WORKSPACE`
was disabled.

- bazelbuild/bazel: Undefined @@rules_java_builtin repository with
  --noenable_workspace option
  bazelbuild/bazel#22754
mbland added a commit to mbland/rules_scala that referenced this issue Oct 5, 2024
Related to bazelbuild#1482, bazelbuild#1618, and bazelbuild#1619. Results from the investigation
documented at:

- bazelbuild#1619 (comment)

Updates `_import_paths()` in `scala_proto_aspect.bzl` to handle
differences `ProtoInfo.proto_source_root` and `ProtoInfo.direct_sources`
values between Bazel 6 and Bazel 7.

Without this change, `_import_paths()` emits incorrect values under
Bazel 7, causing targets containing generated `.proto` inputs to fail,
e.g.  `//test/proto3:test_generated_proto`.

See also:

- Fix paths for sibling repository setup and generated .proto files
  bazelbuild/bazel@6c6c196

- The docstring for `ProtoInfo.proto_source_root` in the Bazel sources:
  https://github.com/bazelbuild/bazel/blob/7.3.2/src/main/starlark/builtins_bzl/common/proto/proto_info.bzl#L155-L172

- Remove incompatible_generated_protos_in_virtual_imports
  bazelbuild/bazel@3efaa32

- Comment from: Generated Protos are no longer considered as
  virtual_imports in Bazel 7
  bazelbuild/bazel#21075 (comment)

---

I cherrypicked this commit into bazelbuild#1618. While it fixed the
`//test/proto3` build failure, it does _not_ fix the hanging
scalapb_workers from the ProtoScalaPBRule aspect.

I'll have to investiate further whether than hang is related to Bazel,
rules_proto, com_google_protobuf, or some mixture thereof. Still,
progress!
mbland added a commit to mbland/rules_scala that referenced this issue Oct 5, 2024
Related to bazelbuild#1482, bazelbuild#1618, and bazelbuild#1619. Results from the investigation
documented at:

- bazelbuild#1619 (comment)

Updates `_import_paths()` in `scala_proto_aspect.bzl` to handle
differences `ProtoInfo.proto_source_root` and `ProtoInfo.direct_sources`
values between Bazel 6 and Bazel 7.

Without this change, `_import_paths()` emits incorrect values under
Bazel 7, causing targets containing generated `.proto` inputs to fail,
e.g.  `//test/proto3:test_generated_proto`.

See also:

- Fix paths for sibling repository setup and generated .proto files
  bazelbuild/bazel@6c6c196

- The docstring for `ProtoInfo.proto_source_root` in the Bazel sources:
  https://github.com/bazelbuild/bazel/blob/7.3.2/src/main/starlark/builtins_bzl/common/proto/proto_info.bzl#L155-L172

- Remove incompatible_generated_protos_in_virtual_imports
  bazelbuild/bazel@3efaa32

- Comment from: Generated Protos are no longer considered as
  virtual_imports in Bazel 7
  bazelbuild/bazel#21075 (comment)

---

I cherrypicked this commit into bazelbuild#1618. While it fixed the
`//test/proto3` build failure, it does _not_ fix the hanging
scalapb_workers from the ProtoScalaPBRule aspect.

I'll have to investiate further whether than hang is related to Bazel,
rules_proto, com_google_protobuf, or some mixture thereof. Still,
progress!
mbland added a commit to mbland/rules_scala that referenced this issue Oct 6, 2024
Related to bazelbuild#1482, bazelbuild#1618, and bazelbuild#1619. Results from the investigation
documented at:

- bazelbuild#1619 (comment)

Updates `_import_paths()` in `scala_proto_aspect.bzl` to handle
differences `ProtoInfo.proto_source_root` and `ProtoInfo.direct_sources`
values between Bazel 6 and Bazel 7.

Without this change, `_import_paths()` emits incorrect values under
Bazel 7, causing targets containing generated `.proto` inputs to fail,
e.g.  `//test/proto3:test_generated_proto`.

See also:

- Fix paths for sibling repository setup and generated .proto files
  bazelbuild/bazel@6c6c196

- The docstring for `ProtoInfo.proto_source_root` in the Bazel sources:
  https://github.com/bazelbuild/bazel/blob/7.3.2/src/main/starlark/builtins_bzl/common/proto/proto_info.bzl#L155-L172

- Remove incompatible_generated_protos_in_virtual_imports
  bazelbuild/bazel@3efaa32

- Comment from: Generated Protos are no longer considered as
  virtual_imports in Bazel 7
  bazelbuild/bazel#21075 (comment)

---

I cherrypicked this commit into bazelbuild#1618. While it fixed the
`//test/proto3` build failure, it does _not_ fix the hanging
scalapb_workers from the ProtoScalaPBRule aspect.

I'll have to investiate further whether than hang is related to Bazel,
rules_proto, com_google_protobuf, or some mixture thereof. Still,
progress!
mbland added a commit to mbland/rules_scala that referenced this issue Oct 7, 2024
Related to bazelbuild#1482, bazelbuild#1618, and bazelbuild#1619. Results from the investigation
documented at:

- bazelbuild#1619 (comment)

Updates `_import_paths()` in `scala_proto_aspect.bzl` to handle
differences `ProtoInfo.proto_source_root` and `ProtoInfo.direct_sources`
values between Bazel 6 and Bazel 7.

Without this change, `_import_paths()` emits incorrect values under
Bazel 7, causing targets containing generated `.proto` inputs to fail,
e.g.  `//test/proto3:test_generated_proto`.

See also:

- Fix paths for sibling repository setup and generated .proto files
  bazelbuild/bazel@6c6c196

- The docstring for `ProtoInfo.proto_source_root` in the Bazel sources:
  https://github.com/bazelbuild/bazel/blob/7.3.2/src/main/starlark/builtins_bzl/common/proto/proto_info.bzl#L155-L172

- Remove incompatible_generated_protos_in_virtual_imports
  bazelbuild/bazel@3efaa32

- Comment from: Generated Protos are no longer considered as
  virtual_imports in Bazel 7
  bazelbuild/bazel#21075 (comment)

---

I cherrypicked this commit into bazelbuild#1618. While it fixed the
`//test/proto3` build failure, it does _not_ fix the hanging
scalapb_workers from the ProtoScalaPBRule aspect.

I'll have to investiate further whether than hang is related to Bazel,
rules_proto, com_google_protobuf, or some mixture thereof. Still,
progress!
mbland added a commit to mbland/rules_scala that referenced this issue Oct 7, 2024
Part of bazelbuild#1482.

Splits the last component off of canonical repo names to produce the
expected repo name.

Without Bzlmod, it returns the original name. With Bzlmod enabled, it
avoids generating output like:

    scala_import(
        name = "_main~scala_deps~io_bazel_rules_scala_scala_compiler",
        jars = ["scala-compiler-2.12.18.jar"],
    )

resulting in errors like:

```
ERROR: .../_main~_repo_rules~io_bazel_rules_scala/scala/BUILD:
no such target '@@_main~scala_deps~io_bazel_rules_scala_scala_compiler//:io_bazel_rules_scala_scala_compiler':
target 'io_bazel_rules_scala_scala_compiler' not declared in package ''
defined by .../_main~scala_deps~io_bazel_rules_scala_scala_compiler/BUILD
and referenced by '@@_main~_repo_rules~io_bazel_rules_scala//scala:default_toolchain_scala_compile_classpath_provider'
```

Also fixes the following error when attaching resources from custom repos to
targets under Bzlmod:

```txt
$ bazel test //test/src/main/scala/scalarules/test/resources:all

1) Scala library depending on resources from external resource-only
  jar::allow to load resources(scalarules.test.resources.ScalaLibResourcesFromExternalDepTest)
  java.lang.NullPointerException
    at scalarules.test.resources.ScalaLibResourcesFromExternalDepTest.get(ScalaLibResourcesFromExternalDepTest.scala:17)
    at scalarules.test.resources.ScalaLibResourcesFromExternalDepTest.$anonfun$new$3(ScalaLibResourcesFromExternalDepTest.scala:11)
    at scalarules.test.resources.ScalaLibResourcesFromExternalDepTest.$anonfun$new$2(ScalaLibResourcesFromExternalDepTest.scala:11)
```

Can be replaced with a future bazel-skylib implementation, if accepted
into that repo.

---

We can't rely on the specific canonical repository name format:

> Repos generated by extensions have canonical names in the form of
> `module_repo_canonical_name+extension_name+repo_name`. For extensions
> hosted in the root module, the `module_repo_canonical_name` part is
> replaced with the string `_main`. Note that the canonical name format is
> not an API you should depend on — it's subject to change at any time.
>
> - https://bazel.build/external/extension#repository_names_and_visibility

The change to no longer encode module versions in canonical repo names in
Bazel 7.1.0 is a recent example of Bazel maintainers altering the format:

- bazelbuild/bazel#21316

And the maintainers recently replaced the `~` delimiter with `+` in the
upcoming Bazel 8 release due to build performance issues on Windows:

- bazelbuild/bazel#22865

This function assumes the only valid `repo_name` characters are letters,
numbers, '_', '-', and '.'. It finds the last character not in this set, and
returns the contents of `name` following this character. This is valid so
long as this condition holds:

- https://github.com/bazelbuild/bazel/blob/7.3.2/src/main/java/com/google/devtools/build/lib/cmdline/RepositoryName.java#L159-L162
mbland added a commit to mbland/rules_scala that referenced this issue Oct 7, 2024
Part of bazelbuild#1482.

Splits the last component off of canonical repo names to produce the
expected repo name.

Without Bzlmod, it returns the original name. With Bzlmod enabled, it
avoids generating output like:

    scala_import(
        name = "_main~scala_deps~io_bazel_rules_scala_scala_compiler",
        jars = ["scala-compiler-2.12.18.jar"],
    )

resulting in errors like:

```
ERROR: .../_main~_repo_rules~io_bazel_rules_scala/scala/BUILD:
no such target '@@_main~scala_deps~io_bazel_rules_scala_scala_compiler//:io_bazel_rules_scala_scala_compiler':
target 'io_bazel_rules_scala_scala_compiler' not declared in package ''
defined by .../_main~scala_deps~io_bazel_rules_scala_scala_compiler/BUILD
and referenced by '@@_main~_repo_rules~io_bazel_rules_scala//scala:default_toolchain_scala_compile_classpath_provider'
```

Also fixes the following error when attaching resources from custom repos to
targets under Bzlmod:

```txt
$ bazel test //test/src/main/scala/scalarules/test/resources:all

1) Scala library depending on resources from external resource-only
  jar::allow to load resources(scalarules.test.resources.ScalaLibResourcesFromExternalDepTest)
  java.lang.NullPointerException
    at scalarules.test.resources.ScalaLibResourcesFromExternalDepTest.get(ScalaLibResourcesFromExternalDepTest.scala:17)
    at scalarules.test.resources.ScalaLibResourcesFromExternalDepTest.$anonfun$new$3(ScalaLibResourcesFromExternalDepTest.scala:11)
    at scalarules.test.resources.ScalaLibResourcesFromExternalDepTest.$anonfun$new$2(ScalaLibResourcesFromExternalDepTest.scala:11)
```

Can be replaced with a future bazel-skylib implementation, if accepted
into that repo.

---

We can't rely on the specific canonical repository name format:

> Repos generated by extensions have canonical names in the form of
> `module_repo_canonical_name+extension_name+repo_name`. For extensions
> hosted in the root module, the `module_repo_canonical_name` part is
> replaced with the string `_main`. Note that the canonical name format is
> not an API you should depend on — it's subject to change at any time.
>
> - https://bazel.build/external/extension#repository_names_and_visibility

The change to no longer encode module versions in canonical repo names in
Bazel 7.1.0 is a recent example of Bazel maintainers altering the format:

- bazelbuild/bazel#21316

And the maintainers recently replaced the `~` delimiter with `+` in the
upcoming Bazel 8 release due to build performance issues on Windows:

- bazelbuild/bazel#22865

This function assumes the only valid `repo_name` characters are letters,
numbers, '_', '-', and '.'. It finds the last character not in this set, and
returns the contents of `name` following this character. This is valid so
long as this condition holds:

- https://github.com/bazelbuild/bazel/blob/7.3.2/src/main/java/com/google/devtools/build/lib/cmdline/RepositoryName.java#L159-L162
liucijus pushed a commit that referenced this issue Oct 8, 2024
Related to #1482, #1618, and #1619. Results from the investigation
documented at:

- #1619 (comment)

Updates `_import_paths()` in `scala_proto_aspect.bzl` to handle
differences `ProtoInfo.proto_source_root` and `ProtoInfo.direct_sources`
values between Bazel 6 and Bazel 7.

Without this change, `_import_paths()` emits incorrect values under
Bazel 7, causing targets containing generated `.proto` inputs to fail,
e.g.  `//test/proto3:test_generated_proto`.

See also:

- Fix paths for sibling repository setup and generated .proto files
  bazelbuild/bazel@6c6c196

- The docstring for `ProtoInfo.proto_source_root` in the Bazel sources:
  https://github.com/bazelbuild/bazel/blob/7.3.2/src/main/starlark/builtins_bzl/common/proto/proto_info.bzl#L155-L172

- Remove incompatible_generated_protos_in_virtual_imports
  bazelbuild/bazel@3efaa32

- Comment from: Generated Protos are no longer considered as
  virtual_imports in Bazel 7
  bazelbuild/bazel#21075 (comment)

---

I cherrypicked this commit into #1618. While it fixed the
`//test/proto3` build failure, it does _not_ fix the hanging
scalapb_workers from the ProtoScalaPBRule aspect.

I'll have to investiate further whether than hang is related to Bazel,
rules_proto, com_google_protobuf, or some mixture thereof. Still,
progress!
@simuons simuons reopened this Oct 8, 2024
mbland added a commit to mbland/rules_scala that referenced this issue Oct 8, 2024
Part of bazelbuild#1482.

Splits the last component off of canonical repo names to produce the
expected repo name.

Without Bzlmod, it returns the original name. With Bzlmod enabled, it
avoids generating output like:

    scala_import(
        name = "_main~scala_deps~io_bazel_rules_scala_scala_compiler",
        jars = ["scala-compiler-2.12.18.jar"],
    )

resulting in errors like:

```
ERROR: .../_main~_repo_rules~io_bazel_rules_scala/scala/BUILD:
no such target '@@_main~scala_deps~io_bazel_rules_scala_scala_compiler//:io_bazel_rules_scala_scala_compiler':
target 'io_bazel_rules_scala_scala_compiler' not declared in package ''
defined by .../_main~scala_deps~io_bazel_rules_scala_scala_compiler/BUILD
and referenced by '@@_main~_repo_rules~io_bazel_rules_scala//scala:default_toolchain_scala_compile_classpath_provider'
```

Also fixes the following error when attaching resources from custom repos to
targets under Bzlmod:

```txt
$ bazel test //test/src/main/scala/scalarules/test/resources:all

1) Scala library depending on resources from external resource-only
  jar::allow to load resources(scalarules.test.resources.ScalaLibResourcesFromExternalDepTest)
  java.lang.NullPointerException
    at scalarules.test.resources.ScalaLibResourcesFromExternalDepTest.get(ScalaLibResourcesFromExternalDepTest.scala:17)
    at scalarules.test.resources.ScalaLibResourcesFromExternalDepTest.$anonfun$new$3(ScalaLibResourcesFromExternalDepTest.scala:11)
    at scalarules.test.resources.ScalaLibResourcesFromExternalDepTest.$anonfun$new$2(ScalaLibResourcesFromExternalDepTest.scala:11)
```

Can be replaced with a future bazel-skylib implementation, if accepted
into that repo.

---

We can't rely on the specific canonical repository name format:

> Repos generated by extensions have canonical names in the form of
> `module_repo_canonical_name+extension_name+repo_name`. For extensions
> hosted in the root module, the `module_repo_canonical_name` part is
> replaced with the string `_main`. Note that the canonical name format is
> not an API you should depend on — it's subject to change at any time.
>
> - https://bazel.build/external/extension#repository_names_and_visibility

The change to no longer encode module versions in canonical repo names in
Bazel 7.1.0 is a recent example of Bazel maintainers altering the format:

- bazelbuild/bazel#21316

And the maintainers recently replaced the `~` delimiter with `+` in the
upcoming Bazel 8 release due to build performance issues on Windows:

- bazelbuild/bazel#22865

This function assumes the only valid `repo_name` characters are letters,
numbers, '_', '-', and '.'. It finds the last character not in this set, and
returns the contents of `name` following this character. This is valid so
long as this condition holds:

- https://github.com/bazelbuild/bazel/blob/7.3.2/src/main/java/com/google/devtools/build/lib/cmdline/RepositoryName.java#L159-L162
@mbland
Copy link
Contributor

mbland commented Oct 11, 2024

A quick update for visibility: I'm very close to having the Bzlmodified rules_scala passing 100% of the tests, down to the last couple of failures:

  • I need to replace the bind() calls from twitter_scrooge with alias() or some such to get test_version.sh to pass.
  • I need to fix the Scala 3.1.2 and Scala 3.3.3 failures for the Scalafmt targets from test/shell/test_cross_build.sh (corresponding to the test3 and library3 test cases in test_scalafmt()).

I also need to get test_lint.sh working again, and update the dt_patches repos for Bzlmod. But the following all pass:

  • test_rules_scala.sh
  • test_reproducibility.sh
  • test_examples.sh
  • test_coverage.sh

Hopefully I can get these fixed up today, and I'll start peeling off the next pull request or two. And thanks to @simuons and @liucijus for reviewing and merging #1619 and #1620 already.

mbland added a commit to mbland/bazel-skylib that referenced this issue Oct 14, 2024
Adds the following macros to work with apparent repo names when running
under Bzlmod.

- `adjust_main_repo_prefix`
- `apparent_repo_label_string`
- `apparent_repo_name`

Originally developed while updating rules_scala to support Bzlmod as
part of bazelbuild/rules_scala#1482.

For examples of their use, see bazelbuild/rules_scala#1621.
mbland added a commit to mbland/rules_scala that referenced this issue Oct 15, 2024
Part of bazelbuild#1482.

These are mostly small changes to make test assertions more flexible
between `WORKSPACE` and Bzlmod runs. For Bzlmod runs:

- Fixed `test_scala_config_content` from `test_scala_config.sh` by
  changing a path from `external/io_bazel_rules_scala_config` to
  `external/*io_bazel_rules_scala_config`.

- Fixed a number of tests by updating expected output messages to allow
  them to start with either `@//` or `@@//`.

- Fixed `test_stamped_target_label_loading` from
  `test/shell/test_strict_dependency.sh` by accommodating the canonical
  `io_bazel_rules_scala_guava` repo name. Also allows for the optional
  current Scala version suffix.

Also made these other important changes:

- Updated all the assertions in `test_helper.sh` to use Bash builtin regex
  matching via `_expect_failure_with_messages` instead of `grep`. This
  allows the expected message patterns to use full regular expressions
  while avoiding forking a new process. This new function helped reduce
  duplication in that file at the same time.

- Added `--repo_env="SCALA_VERSION=..."` to each test script called from
  `./test_coverage.sh`, and set `SCALA_VERSION` to 2.12.19 in each of
  these files. Using other Scala versions technically works, but the
  output is slightly different, causing the `diff` commands in the test
  cases to fail.

- Updated `test_version.sh` to copy the top level `.bazelversion` file
  into its test repo.

- Changed how `test_version.sh` handles injecting `twitter_scrooge`
  repos into the `WORKSPACE` file. This will make it easy to do the
  equivalent for `MODULE.bazel` when the time comes.

Also includes a few minor, opportunistic formatting cleanups.
mbland added a commit to mbland/rules_scala that referenced this issue Oct 17, 2024
Part of bazelbuild#1482.

Bumps every Scala version up to use protobuf-java:4.28.2, to
preemptively avoid the following error produced by upcoming Scalafmt
updates:

```txt
$ bazel test --repo_env=SCALA_VERSION=2.11.12 //test/scalafmt/...

INFO: Analyzed 9 targets (80 packages loaded, 3307 targets configured).
ERROR: .../test/scalafmt/BUILD:43:20: ScalaFmt
  test/scalafmt/test/scalafmt/formatted/formatted-test.scala.fmt.output
  failed: Worker process did not return a WorkResponse:

---8<---8<--- Start of log, file at .../bazel-workers/worker-134-ScalaFmt.log ---8<---8<---
Exception in thread "main" java.lang.NoSuchMethodError:
  'boolean com.google.protobuf.GeneratedMessageV3.isStringEmpty(java.lang.Object)'
  [ ...snip... ]
```

This issue seemed to suggest a library bump may fix it, and it did:

- protocolbuffers/protobuf#9236

Fortunately, even though this is a major version bump from 3.10.0 to
4.28.2, there were no compatibility issues, per:

> Protobuf major version releases may also be backwards-compatible with
> the last release of the previous major version. See the release notice
> for more details.
>
> - https://github.com/protocolbuffers/protobuf/tree/main/java#compatibility-notice
mbland added a commit to mbland/rules_scala that referenced this issue Oct 18, 2024
Use custom repo rule to generate @scalafmt_default

Part of bazelbuild#1482.

Replaces `native.new_local_repository` in `scalafmt_default_config` with
the new `scalafmt_config` repo rule. Resolves an incompatibility between
Bazel 6.5.0 and 7.3.2, while creating a much smaller `@scalafmt_default`
repo.

The problem is that under Bazel 7.3.2, calling `scalafmt_default_config`
from a module extension produces this error:

```txt
$ bazel test //test/scalafmt/...

ERROR: Traceback (most recent call last):
  File "/Users/mbland/src/bazelbuild/rules_scala-bzlmod/scala/extensions/deps.bzl",
    line 218, column 32, in _scala_deps_impl
    scalafmt_default_config()
  File "/Users/mbland/src/bazelbuild/rules_scala-bzlmod/scala/scalafmt/scalafmt_repositories.bzl",
    line 18, column 32, in scalafmt_default_config
    native.new_local_repository(

Error in new_local_repository: The native module can be accessed only
  from a BUILD thread. Wrap the function in a macro and call it from a
  BUILD file
```

Ostensibly the solution is to replace `native.new_local_repository`
with:

```py
load(
  "@bazel_tools//tools/build_defs/repo:local.bzl",
  "new_local_repository",
)
```

However, `local.bzl` isn't available in Bazel 6.5.0:

```txt
$ bazel test //test/scalafmt/...

ERROR: Error computing the main repository mapping:
  at .../scala/scalafmt/scalafmt_repositories.bzl:8:6:
cannot load '@bazel_tools//tools/build_defs/repo:local.bzl':
  no such file
```

The new `scalafmt_config` repository rule works under both Bazel 6.5.0
and 7.3.2. Also, it symlinks only the single Scalafmt config file,
instead of every top level file and directory in the project, as
`new_local_repository` did.
mbland added a commit to mbland/rules_scala that referenced this issue Oct 18, 2024
Part of bazelbuild#1482.

Replaces `native.new_local_repository` in `scalafmt_default_config` with
the new `scalafmt_config` repo rule. Resolves an incompatibility between
Bazel 6.5.0 and 7.3.2, while creating a much smaller `@scalafmt_default`
repo. Also updates the `native.register_toolchains` call with a
stringified `Label` to remove the hardcoding of `@io_bazel_rules_scala`.

The problem is that under Bazel 7.3.2, calling `scalafmt_default_config`
from a module extension produces this error:

```txt
$ bazel test //test/scalafmt/...

ERROR: Traceback (most recent call last):
  File ".../scala/extensions/deps.bzl",
    line 218, column 32, in _scala_deps_impl
    scalafmt_default_config()
  File ".../scala/scalafmt/scalafmt_repositories.bzl",
    line 18, column 32, in scalafmt_default_config
    native.new_local_repository(

Error in new_local_repository: The native module can be accessed only
  from a BUILD thread. Wrap the function in a macro and call it from a
  BUILD file
```

Ostensibly the solution is to replace `native.new_local_repository`
with:

```py
load(
  "@bazel_tools//tools/build_defs/repo:local.bzl",
  "new_local_repository",
)
```

However, `local.bzl` isn't available in Bazel 6.5.0:

```txt
$ bazel test //test/scalafmt/...

ERROR: Error computing the main repository mapping:
  at .../scala/scalafmt/scalafmt_repositories.bzl:8:6:
cannot load '@bazel_tools//tools/build_defs/repo:local.bzl':
  no such file
```

The new `scalafmt_config` repository rule works under both Bazel 6.5.0
and 7.3.2. Also, it symlinks only the single Scalafmt config file,
instead of every top level file and directory in the project, as
`new_local_repository` did.
@mbland
Copy link
Contributor

mbland commented Oct 18, 2024

@liucijus @simuons Let me know if I'm sending too many PRs too quickly and if I should start holding some back. Up to this point, they're largely orthogonal and can go in in any order, but I'm getting to the point where some changes will depend on earlier PRs. (I can always rebase the future PRs, if you don't mind me running ahead.)

Also, please check out #1625 that I filed today. We're kind of stuck with the "Bazel green head" job failing in a way that breaks our builds due to the problem I described there.

mbland added a commit to mbland/rules_scala that referenced this issue Oct 18, 2024
Part of bazelbuild#1482.

Bumps every Scala version up to use protobuf-java:4.28.2, to
preemptively avoid the following error produced by upcoming Scalafmt
updates:

```txt
$ bazel test --repo_env=SCALA_VERSION=2.11.12 //test/scalafmt/...

INFO: Analyzed 9 targets (80 packages loaded, 3307 targets configured).
ERROR: .../test/scalafmt/BUILD:43:20: ScalaFmt
  test/scalafmt/test/scalafmt/formatted/formatted-test.scala.fmt.output
  failed: Worker process did not return a WorkResponse:

---8<---8<--- Start of log, file at .../bazel-workers/worker-134-ScalaFmt.log ---8<---8<---
Exception in thread "main" java.lang.NoSuchMethodError:
  'boolean com.google.protobuf.GeneratedMessageV3.isStringEmpty(java.lang.Object)'
  [ ...snip... ]
```

This issue seemed to suggest a library bump may fix it, and it did:

- protocolbuffers/protobuf#9236

Fortunately, even though this is a major version bump from 3.10.0 to
4.28.2, there were no compatibility issues, per:

> Protobuf major version releases may also be backwards-compatible with
> the last release of the previous major version. See the release notice
> for more details.
>
> - https://github.com/protocolbuffers/protobuf/tree/main/java#compatibility-notice
mbland added a commit to mbland/rules_scala that referenced this issue Oct 19, 2024
Part of bazelbuild#1482.

Ensures Bazelisk uses the same Bazel version in every workspace in the
project.

I discovered that the workspaces under the main workspace were running
the latest mainline Blaze release I had installed. Now all the
workspaces are synchronized on the same Bazel version.

As noted in the `tools/sync-bazelversion.sh` comments, symlinks or an
`import ../.bazelversion` mechanism would be preferable. However, the
former can surprise Windows users, and Bazelisk doesn't support the
latter.
mbland added a commit to mbland/rules_scala that referenced this issue Oct 20, 2024
Part of bazelbuild#1482.

Ensures Bazelisk uses the same Bazel version in every workspace in the
project.

I discovered that the workspaces under the main workspace were running
the latest mainline Blaze release I had installed. Now all the
workspaces are synchronized on the same Bazel version.

As noted in the `tools/sync-bazelversion.sh` comments, symlinks or an
`import ../.bazelversion` mechanism would be preferable. However, the
former can surprise Windows users, and Bazelisk doesn't support the
latter.
mbland added a commit to mbland/rules_scala that referenced this issue Oct 20, 2024
Part of bazelbuild#1482. Ensures that all Scalafmt targets succeed under every
supported Scala version.

Scala 2.11 remains on Scalafmt 2.7.5, since there's not a more recent
compatible version.

Between 3.0.0 and 3.8.3, Scalafmt's `FileOps` moved packages,
`Config.fromHoconFile` moved to `ScalafmtConfig`, and the methods now
take a `java.nio.file.Path` parameter. As a result, this required
extracting a thin `ScalafmtAdapter` object, with one implementation for
Scala 2.11 and Scalafmt 2.7.5, and one for Scalafmt 3.8.3.

This change also adds the file path to the `Scalafmt.format()` call,
allowing error messages to show the actual file path instead of
`<input>`.

Also removes some `@io_bazel_rules_scala` references to make the
internal implementation less dependendent on that name. This will allow
Bzlmod clients to use `rules_scala` in a `bazel_dep()` without setting
`repo_name = "io_bazel_rules_scala"`.

---

Scalafmt 3.0.0 works with the current default Scala version 2.12.19,
but breaks under Scala 2.13.14:

```txt
$ bazel test --repo_env=SCALA_VERSION=2.13.14 //test/scalafmt/...

ERROR: .../test/scalafmt/BUILD:49:20:
  ScalaFmt test/scalafmt/test/scalafmt/unformatted/unformatted-test.scala.fmt.output
  failed: (Exit 1): scalafmt failed: error executing command
  (from target //test/scalafmt:unformatted-test)
  bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/scala/scalafmt/scalafmt
  '--jvm_flag=-Dfile.encoding=UTF-8' ... (remaining 1 argument skipped)

java.util.NoSuchElementException: last of empty IndexedSeq
    at scala.collection.IndexedSeqOps.last(IndexedSeq.scala:110)
    at scala.collection.IndexedSeqOps.last$(IndexedSeq.scala:105)
    at scala.meta.tokens.Tokens.last(Tokens.scala:29)
    at org.scalafmt.internal.Router.$anonfun$getSplitsImpl$90(Router.scala:707)
    at scala.Option.map(Option.scala:242)
    at org.scalafmt.internal.Router.getSplitsImpl(Router.scala:706)
    at org.scalafmt.internal.Router.getSplits(Router.scala:2314)
    at org.scalafmt.internal.BestFirstSearch.$anonfun$routes$1(BestFirstSearch.scala:38)
    [ ...snip... ]
```

This matches:

- scala/community-build#1680

Which mentions apparent fixes in:

- scalameta/scalameta#3235
- scalameta/scalafmt#3581

So the fix was to upgrade the Scalafmt version. That said, I held its
Scalameta dependencies to 4.9.9 per the following link, even though
4.10.2 is out now.

- https://mvnrepository.com/artifact/org.scalameta/scalafmt-core_2.13/3.8.3

---

This change updates Scalameta to version 4.9.9 because between 4.9.9 and
4.10.2, Scalafmt breaks Scala 3 file formatting by unindenting some code
that it shouldn't. Or, our usage of it breaks somehow; I can't find any
open or closed issues in the Scalameta project that matches what happes
in rules_scala. (Perhaps I can file one eventually.)

- https://github.com/scalameta/scalafmt/issues

The solution was to keep the Scalameta dependencies at 4.9.9. FWIW, this
was one of the most time consuming bugs to pinpoint and rectify in the
entire Bzlmodification process.

This was the failing command inside `test_cross_version`:

```txt
$ bazel run //scalafmt:unformatted-test3.format-test

INFO: Analyzed target //scalafmt:unformatted-test3.format-test
  (0 packages loaded, 0 targets configured).
INFO: From ScalaFmt scalafmt/scalafmt/unformatted/unformatted-test3.scala.fmt.output:
```

The `test_cross_version/scalafmt/unformatted/unformatted-test3.scala`
file formatted by this test target looks like this:

```scala
import org.scalatest.flatspec._

class Test extends
  AnyFlatSpec:

        "Test"  should  "be formatted" in  {
     assert(true)
        }
```

I hacked `ScalaWorker.format()` to print the `code` variable, and could
see that  after the first `Scalafmt.format()` pass, the code looks like
this:

```scala
import org.scalatest.flatspec._

class Test extends AnyFlatSpec:

"Test" should "be formatted" in {
  assert(true)
}
```

Since the result doesn't match the original code, it tries to call
`Scalafmt.format()` again on this "formatted" code with the incorrect
indentation. That's when we get the following, which doesn't look
anything like the original file:

```txt
Unable to format file due to bug in scalafmt
scalafmt/unformatted/unformatted-test3.scala:3:
  error: [dialect scala3 [with overrides]] `;` expected but `:` found
class Test extends AnyFlatSpec:
                              ^
```

---

As it turns out, bumping to com.google.protobuf:protobuf-java:4.28.2
alone breaks the bump to Scalafmt 3.8.3. Then bumping to rules_proto
6.0.2, with the separate protobuf v21.7, fixes it, presumably by
recompiling `protoc`.

The in-between breakage happened in the `test_cross_build` project:

```txt
$ bazel build //scalafmt:formatted-binary2

INFO: Analyzed target //scalafmt:formatted-binary2
  (4 packages loaded, 64 targets configured).
ERROR: .../test_cross_build/scalafmt/BUILD:59:22:
  ScalaFmt scalafmt/scalafmt/formatted/formatted-binary2.scala.fmt.output
  failed: Worker process did not return a WorkResponse:

---8<---8<--- Start of log, file at .../bazel-workers/worker-3-ScalaFmt.log ---8<---8<---
Exception in thread "main" java.lang.NoSuchMethodError:
  'void com.google.devtools.build.lib.worker.WorkerProtocol$WorkRequest.makeExtensionsImmutable()'
    at com.google.devtools.build.lib.worker.WorkerProtocol$WorkRequest.<init>(WorkerProtocol.java:1029)
    at com.google.devtools.build.lib.worker.WorkerProtocol$WorkRequest.<init>(WorkerProtocol.java:922)
    at com.google.devtools.build.lib.worker.WorkerProtocol$WorkRequest$1.parsePartialFrom(WorkerProtocol.java:2482)
    at com.google.devtools.build.lib.worker.WorkerProtocol$WorkRequest$1.parsePartialFrom(WorkerProtocol.java:2476)
    at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:192)
    at com.google.protobuf.AbstractParser.parsePartialDelimitedFrom(AbstractParser.java:232)
    at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:244)
    at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:249)
    at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:25)
    at com.google.protobuf.GeneratedMessage.parseDelimitedWithIOException(GeneratedMessage.java:367)
    at com.google.devtools.build.lib.worker.WorkerProtocol$WorkRequest.parseDelimitedFrom(WorkerProtocol.java:1438)
    at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:81)
    at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
    at io.bazel.rules_scala.scalafmt.ScalafmtWorker$.main(ScalafmtWorker.scala:12)
    at io.bazel.rules_scala.scalafmt.ScalafmtWorker.main(ScalafmtWorker.scala)
---8<---8<--- End of log ---8<---8<---
Target //scalafmt:formatted-binary2 failed to build
```
mbland added a commit to mbland/rules_scala that referenced this issue Oct 20, 2024
Part of bazelbuild#1482. Ensures that all Scalafmt targets succeed under every
supported Scala version.

Scala 2.11 remains on Scalafmt 2.7.5, since there's not a more recent
compatible version.

Between 3.0.0 and 3.8.3, Scalafmt's `FileOps` moved packages,
`Config.fromHoconFile` moved to `ScalafmtConfig`, and the methods now
take a `java.nio.file.Path` parameter. As a result, this required
extracting a thin `ScalafmtAdapter` object, with one implementation for
Scala 2.11 and Scalafmt 2.7.5, and one for Scalafmt 3.8.3.

This change also adds the file path to the `Scalafmt.format()` call,
allowing error messages to show the actual file path instead of
`<input>`.

Also removes some `@io_bazel_rules_scala` references to make the
internal implementation less dependendent on that name. This will allow
Bzlmod clients to use `rules_scala` in a `bazel_dep()` without setting
`repo_name = "io_bazel_rules_scala"`.

---

Scalafmt 3.0.0 works with the current default Scala version 2.12.19,
but breaks under Scala 2.13.14:

```txt
$ bazel test --repo_env=SCALA_VERSION=2.13.14 //test/scalafmt/...

ERROR: .../test/scalafmt/BUILD:49:20:
  ScalaFmt test/scalafmt/test/scalafmt/unformatted/unformatted-test.scala.fmt.output
  failed: (Exit 1): scalafmt failed: error executing command
  (from target //test/scalafmt:unformatted-test)
  bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/scala/scalafmt/scalafmt
  '--jvm_flag=-Dfile.encoding=UTF-8' ... (remaining 1 argument skipped)

java.util.NoSuchElementException: last of empty IndexedSeq
    at scala.collection.IndexedSeqOps.last(IndexedSeq.scala:110)
    at scala.collection.IndexedSeqOps.last$(IndexedSeq.scala:105)
    at scala.meta.tokens.Tokens.last(Tokens.scala:29)
    at org.scalafmt.internal.Router.$anonfun$getSplitsImpl$90(Router.scala:707)
    at scala.Option.map(Option.scala:242)
    at org.scalafmt.internal.Router.getSplitsImpl(Router.scala:706)
    at org.scalafmt.internal.Router.getSplits(Router.scala:2314)
    at org.scalafmt.internal.BestFirstSearch.$anonfun$routes$1(BestFirstSearch.scala:38)
    [ ...snip... ]
```

This matches:

- scala/community-build#1680

Which mentions apparent fixes in:

- scalameta/scalameta#3235
- scalameta/scalafmt#3581

So the fix was to upgrade the Scalafmt version. That said, I held its
Scalameta dependencies to 4.9.9 per the following link, even though
4.10.2 is out now.

- https://mvnrepository.com/artifact/org.scalameta/scalafmt-core_2.13/3.8.3

---

This change updates Scalameta to version 4.9.9 because between 4.9.9 and
4.10.2, Scalafmt breaks Scala 3 file formatting by unindenting some code
that it shouldn't. Or, our usage of it breaks somehow; I can't find any
open or closed issues in the Scalameta project that matches what happes
in rules_scala. (Perhaps I can file one eventually.)

- https://github.com/scalameta/scalafmt/issues

The solution was to keep the Scalameta dependencies at 4.9.9. FWIW, this
was one of the most time consuming bugs to pinpoint and rectify in the
entire Bzlmodification process.

This was the failing command inside `test_cross_version`:

```txt
$ bazel run //scalafmt:unformatted-test3.format-test

INFO: Analyzed target //scalafmt:unformatted-test3.format-test
  (0 packages loaded, 0 targets configured).
INFO: From ScalaFmt scalafmt/scalafmt/unformatted/unformatted-test3.scala.fmt.output:
```

The `test_cross_version/scalafmt/unformatted/unformatted-test3.scala`
file formatted by this test target looks like this:

```scala
import org.scalatest.flatspec._

class Test extends
  AnyFlatSpec:

        "Test"  should  "be formatted" in  {
     assert(true)
        }
```

I hacked `ScalaWorker.format()` to print the `code` variable, and could
see that  after the first `Scalafmt.format()` pass, the code looks like
this:

```scala
import org.scalatest.flatspec._

class Test extends AnyFlatSpec:

"Test" should "be formatted" in {
  assert(true)
}
```

Since the result doesn't match the original code, it tries to call
`Scalafmt.format()` again on this "formatted" code with the incorrect
indentation. That's when we get the following, which doesn't look
anything like the original file:

```txt
Unable to format file due to bug in scalafmt
scalafmt/unformatted/unformatted-test3.scala:3:
  error: [dialect scala3 [with overrides]] `;` expected but `:` found
class Test extends AnyFlatSpec:
                              ^
```

---

As it turns out, bumping to com.google.protobuf:protobuf-java:4.28.2
alone breaks the bump to Scalafmt 3.8.3. Then bumping to rules_proto
6.0.2, with the separate protobuf v21.7, fixes it, presumably by
recompiling `protoc`.

The in-between breakage happened in the `test_cross_build` project:

```txt
$ bazel build //scalafmt:formatted-binary2

INFO: Analyzed target //scalafmt:formatted-binary2
  (4 packages loaded, 64 targets configured).
ERROR: .../test_cross_build/scalafmt/BUILD:59:22:
  ScalaFmt scalafmt/scalafmt/formatted/formatted-binary2.scala.fmt.output
  failed: Worker process did not return a WorkResponse:

---8<---8<--- Start of log, file at .../bazel-workers/worker-3-ScalaFmt.log ---8<---8<---
Exception in thread "main" java.lang.NoSuchMethodError:
  'void com.google.devtools.build.lib.worker.WorkerProtocol$WorkRequest.makeExtensionsImmutable()'
    at com.google.devtools.build.lib.worker.WorkerProtocol$WorkRequest.<init>(WorkerProtocol.java:1029)
    at com.google.devtools.build.lib.worker.WorkerProtocol$WorkRequest.<init>(WorkerProtocol.java:922)
    at com.google.devtools.build.lib.worker.WorkerProtocol$WorkRequest$1.parsePartialFrom(WorkerProtocol.java:2482)
    at com.google.devtools.build.lib.worker.WorkerProtocol$WorkRequest$1.parsePartialFrom(WorkerProtocol.java:2476)
    at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:192)
    at com.google.protobuf.AbstractParser.parsePartialDelimitedFrom(AbstractParser.java:232)
    at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:244)
    at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:249)
    at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:25)
    at com.google.protobuf.GeneratedMessage.parseDelimitedWithIOException(GeneratedMessage.java:367)
    at com.google.devtools.build.lib.worker.WorkerProtocol$WorkRequest.parseDelimitedFrom(WorkerProtocol.java:1438)
    at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:81)
    at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
    at io.bazel.rules_scala.scalafmt.ScalafmtWorker$.main(ScalafmtWorker.scala:12)
    at io.bazel.rules_scala.scalafmt.ScalafmtWorker.main(ScalafmtWorker.scala)
---8<---8<--- End of log ---8<---8<---
Target //scalafmt:formatted-binary2 failed to build
```
mbland added a commit to mbland/rules_scala that referenced this issue Oct 20, 2024
Part of bazelbuild#1482.

Bumps every Scala version up to use protobuf-java:4.28.2, to
preemptively avoid the following error produced by upcoming Scalafmt
updates:

```txt
$ bazel test --repo_env=SCALA_VERSION=2.11.12 //test/scalafmt/...

INFO: Analyzed 9 targets (80 packages loaded, 3307 targets configured).
ERROR: .../test/scalafmt/BUILD:43:20: ScalaFmt
  test/scalafmt/test/scalafmt/formatted/formatted-test.scala.fmt.output
  failed: Worker process did not return a WorkResponse:

---8<---8<--- Start of log, file at .../bazel-workers/worker-134-ScalaFmt.log ---8<---8<---
Exception in thread "main" java.lang.NoSuchMethodError:
  'boolean com.google.protobuf.GeneratedMessageV3.isStringEmpty(java.lang.Object)'
  [ ...snip... ]
```

This issue seemed to suggest a library bump may fix it, and it did:

- protocolbuffers/protobuf#9236

Fortunately, even though this is a major version bump from 3.10.0 to
4.28.2, there were no compatibility issues, per:

> Protobuf major version releases may also be backwards-compatible with
> the last release of the previous major version. See the release notice
> for more details.
>
> - https://github.com/protocolbuffers/protobuf/tree/main/java#compatibility-notice
mbland added a commit to mbland/rules_scala that referenced this issue Oct 20, 2024
Part of bazelbuild#1482.

Ensures Bazelisk uses the same Bazel version in every workspace in the
project.

I discovered that the workspaces under the main workspace were running
the latest mainline Blaze release I had installed. Now all the
workspaces are synchronized on the same Bazel version.

As noted in the `tools/sync-bazelversion.sh` comments, symlinks or an
`import ../.bazelversion` mechanism would be preferable. However, the
former can surprise Windows users, and Bazelisk doesn't support the
latter.
mbland added a commit to mbland/rules_scala that referenced this issue Oct 20, 2024
Part of bazelbuild#1482. Enabled by bumping rules_proto to 6.0.2, with the
separate protobuf repo and updating the protoc-bridge and grpc libraries
to their latest versions.

The updated versions are:

- abseil-cpp 20240722.0
- protobuf v28.2
- rules_cc 0.0.13

NOTE: This doesn't work with Bazel 6.x, because it doesn't support
C++14, required by abseil-cpp versions after 20220623.1. This holds even
after the rules_cc bump.

Also note that the `ProtoScalaPBRule` aspect workers hang for the time
being:

```txt
$ USE_BAZEL_VERSION=7.3.2 bazel build //test/...

ERROR: .../external/com_google_protobuf/src/google/protobuf/BUILD.bazel:127:14:
  ProtoScalaPBRule external/com_google_protobuf/src/google/protobuf/wrappers_proto_jvm_extra_protobuf_generator_scalapb.srcjar
  failed: (Exit 1): scalapb_worker failed:
  error executing ProtoScalaPBRule command
  (from target @@com_google_protobuf//src/google/protobuf:wrappers_proto)
  bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker
    ... (remaining 2 arguments skipped)

--jvm_extra_protobuf_generator_out: java.lang.NoSuchMethodError:
  'java.lang.Object com.google.protobuf.DescriptorProtos$FieldOptions.getExtension(com.google.protobuf.GeneratedMessage$GeneratedExtension)'
  java.lang.RuntimeException: Exit with code 1
    at scala.sys.package$.error(package.scala:30)
    at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
    at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
    at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
    at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
    at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)

ERROR: .../external/com_google_protobuf/src/google/protobuf/BUILD.bazel:127:14
  scala @com_google_protobuf//src/google/protobuf:wrappers_proto
  failed: (Exit 1): scalapb_worker failed:
  error executing ProtoScalaPBRule command
  (from target @@com_google_protobuf//src/google/protobuf:wrappers_proto)
  bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker
    ... (remaining 2 arguments skipped)
```
mbland added a commit to mbland/rules_scala that referenced this issue Oct 20, 2024
Part of bazelbuild#1482. Enabled by bumping rules_proto to 6.0.2, with the
separate protobuf repo and updating the protoc-bridge and grpc libraries
to their latest versions.

The updated versions are:

- abseil-cpp 20240722.0
- protobuf v28.2
- rules_cc 0.0.13

NOTE: This doesn't work with Bazel 6.x, because it doesn't support
C++14, required by abseil-cpp versions after 20220623.1. This holds even
after the rules_cc bump.

Also note that the `ProtoScalaPBRule` aspect workers fail for the time
being. The `validator.validateFiles()` call from
`ExtraProtobufGenerator.handleCodeGeneratorRequest()` throws the
following exception:

```txt
$ USE_BAZEL_VERSION=7.3.2 bazel build //test/...

ERROR: .../external/com_google_protobuf/src/google/protobuf/BUILD.bazel:127:14:
  ProtoScalaPBRule external/com_google_protobuf/src/google/protobuf/wrappers_proto_jvm_extra_protobuf_generator_scalapb.srcjar
  failed: (Exit 1): scalapb_worker failed:
  error executing ProtoScalaPBRule command
  (from target @@com_google_protobuf//src/google/protobuf:wrappers_proto)
  bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker
    ... (remaining 2 arguments skipped)

--jvm_extra_protobuf_generator_out: java.lang.NoSuchMethodError:
  'java.lang.Object com.google.protobuf.DescriptorProtos$FieldOptions.getExtension(com.google.protobuf.GeneratedMessage$GeneratedExtension)'
  java.lang.RuntimeException: Exit with code 1
    at scala.sys.package$.error(package.scala:30)
    at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
    at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
    at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
    at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
    at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)

ERROR: .../external/com_google_protobuf/src/google/protobuf/BUILD.bazel:127:14
  scala @com_google_protobuf//src/google/protobuf:wrappers_proto
  failed: (Exit 1): scalapb_worker failed:
  error executing ProtoScalaPBRule command
  (from target @@com_google_protobuf//src/google/protobuf:wrappers_proto)
  bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker
    ... (remaining 2 arguments skipped)
```
mbland added a commit to mbland/rules_scala that referenced this issue Oct 21, 2024
Part of bazelbuild#1482.

Bumps every Scala version up to use protobuf-java:4.28.2, to
preemptively avoid the following error produced by upcoming Scalafmt
updates:

```txt
$ bazel test --repo_env=SCALA_VERSION=2.11.12 //test/scalafmt/...

INFO: Analyzed 9 targets (80 packages loaded, 3307 targets configured).
ERROR: .../test/scalafmt/BUILD:43:20: ScalaFmt
  test/scalafmt/test/scalafmt/formatted/formatted-test.scala.fmt.output
  failed: Worker process did not return a WorkResponse:

---8<---8<--- Start of log, file at .../bazel-workers/worker-134-ScalaFmt.log ---8<---8<---
Exception in thread "main" java.lang.NoSuchMethodError:
  'boolean com.google.protobuf.GeneratedMessageV3.isStringEmpty(java.lang.Object)'
  [ ...snip... ]
```

This issue seemed to suggest a library bump may fix it, and it did:

- protocolbuffers/protobuf#9236

Fortunately, even though this is a major version bump from 3.10.0 to
4.28.2, there were no compatibility issues, per:

> Protobuf major version releases may also be backwards-compatible with
> the last release of the previous major version. See the release notice
> for more details.
>
> - https://github.com/protocolbuffers/protobuf/tree/main/java#compatibility-notice
mbland added a commit to mbland/rules_scala that referenced this issue Oct 21, 2024
Part of bazelbuild#1482.

Ensures Bazelisk uses the same Bazel version in every workspace in the
project.

I discovered that the workspaces under the main workspace were running
the latest mainline Blaze release I had installed. Now all the
workspaces are synchronized on the same Bazel version.

As noted in the `tools/sync-bazelversion.sh` comments, symlinks or an
`import ../.bazelversion` mechanism would be preferable. However, the
former can surprise Windows users, and Bazelisk doesn't support the
latter.
mbland added a commit to mbland/rules_scala that referenced this issue Oct 21, 2024
Part of bazelbuild#1482. Enabled by bumping rules_proto to 6.0.2, with the
separate protobuf repo and updating the protoc-bridge and grpc libraries
to their latest versions.

The updated versions are:

- abseil-cpp 20240722.0
- protobuf v28.2
- rules_cc 0.0.13

NOTE: This doesn't work with Bazel 6.x, because it doesn't support
C++14, required by abseil-cpp versions after 20220623.1. This holds even
after the rules_cc bump.

Also note that the `ProtoScalaPBRule` aspect workers fail for the time
being. The `validator.validateFiles()` call from
`ExtraProtobufGenerator.handleCodeGeneratorRequest()` throws the
following exception:

```txt
$ USE_BAZEL_VERSION=7.3.2 bazel build //test/...

ERROR: .../test/proto/BUILD:165:14:
  ProtoScalaPBRule test/proto/standalone_proto_strip_import_prefix_package_jvm_extra_protobuf_generator_scalapb.srcjar
  failed: (Exit 1): scalapb_worker failed:
  error executing ProtoScalaPBRule command
  (from target //test/proto:standalone_proto_strip_import_prefix_package)
  bazel-bin/src/scala/scripts/scalapb_worker
    ... (remaining 2 arguments skipped)

--jvm_extra_protobuf_generator_out: java.lang.NoSuchMethodError:
'java.lang.Object com.google.protobuf.DescriptorProtos$MessageOptions.getExtension(com.google.protobuf.GeneratedMessage$GeneratedExtension)'
    at scalapb.compiler.DescriptorImplicits$ExtendedMessageDescriptor.messageOptions(DescriptorImplicits.scala:532)
    at scalapb.compiler.DescriptorImplicits$ExtendedMessageDescriptor.sealedOneOfExtendsCount(DescriptorImplicits.scala:578)
    at scalapb.compiler.ProtoValidation.validateMessage(ProtoValidation.scala:107)
    at scalapb.compiler.ProtoValidation.$anonfun$validateFile$2(ProtoValidation.scala:17)
    at scalapb.compiler.ProtoValidation.$anonfun$validateFile$2$adapted(ProtoValidation.scala:17)
    at scala.collection.Iterator.foreach(Iterator.scala:943)
    at scala.collection.Iterator.foreach$(Iterator.scala:943)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
    at scala.collection.IterableLike.foreach(IterableLike.scala:74)
    at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
    at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
    at scalapb.compiler.ProtoValidation.validateFile(ProtoValidation.scala:17)
    at scalapb.compiler.ProtoValidation.$anonfun$validateFiles$1(ProtoValidation.scala:10)
    at scalapb.compiler.ProtoValidation.$anonfun$validateFiles$1$adapted(ProtoValidation.scala:10)
    at scala.collection.immutable.Stream.foreach(Stream.scala:533)
    at scalapb.compiler.ProtoValidation.validateFiles(ProtoValidation.scala:10)
    at scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator$.handleCodeGeneratorRequest(ExtraProtobufGenerator.scala:65)
    at scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator$.run(ExtraProtobufGenerator.scala:47)
    at protocbridge.frontend.PluginFrontend$.$anonfun$runWithBytes$1(PluginFrontend.scala:51)
    at scala.util.Try$.apply(Try.scala:213)
    at protocbridge.frontend.PluginFrontend$.runWithBytes(PluginFrontend.scala:51)
    at protocbridge.frontend.PluginFrontend$.runWithInputStream(PluginFrontend.scala:121)
    at protocbridge.frontend.PosixPluginFrontend$.$anonfun$prepare$2(PosixPluginFrontend.scala:40)
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
    at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1$$anon$2.block(ExecutionContextImpl.scala:75)
    at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3118)
    at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1.blockOn(ExecutionContextImpl.scala:87)
    at scala.concurrent.package$.blocking(package.scala:146)
    at protocbridge.frontend.PosixPluginFrontend$.$anonfun$prepare$1(PosixPluginFrontend.scala:38)
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
    at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
    at scala.util.Success.$anonfun$map$1(Try.scala:255)
    at scala.util.Success.map(Try.scala:213)
    at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
    at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:42)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:74)
    at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

java.lang.RuntimeException: Exit with code 1
    at scala.sys.package$.error(package.scala:30)
    at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
    at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
    at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
    at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
    at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)

ERROR: .../test/proto/BUILD:165:14
  scala @//test/proto:standalone_proto_strip_import_prefix_package
  failed: (Exit 1): scalapb_worker failed:
  error executing ProtoScalaPBRule command
  (from target //test/proto:standalone_proto_strip_import_prefix_package)
  bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker
    ... (remaining 2 arguments skipped)
```
mbland added a commit to mbland/rules_scala that referenced this issue Oct 21, 2024
Part of bazelbuild#1482. Enabled by bumping rules_proto to 6.0.2, with the
separate protobuf repo and updating the protoc-bridge and grpc libraries
to their latest versions.

The updated versions are:

- abseil-cpp 20240722.0
- protobuf v28.2
- rules_cc 0.0.13

NOTE: This doesn't work with Bazel 6.x, because it doesn't support
C++14, required by abseil-cpp versions after 20220623.1. This holds even
after the rules_cc bump.

Also note that the `ProtoScalaPBRule` aspect workers fail for the time
being. The `validator.validateFiles()` call from
`ExtraProtobufGenerator.handleCodeGeneratorRequest()` throws the
following exception:

```txt
$ USE_BAZEL_VERSION=7.3.2 bazel build //test/...

ERROR: .../test/proto/BUILD:165:14:
  ProtoScalaPBRule test/proto/standalone_proto_strip_import_prefix_package_jvm_extra_protobuf_generator_scalapb.srcjar
  failed: (Exit 1): scalapb_worker failed:
  error executing ProtoScalaPBRule command
  (from target //test/proto:standalone_proto_strip_import_prefix_package)
  bazel-bin/src/scala/scripts/scalapb_worker
    ... (remaining 2 arguments skipped)

--jvm_extra_protobuf_generator_out: java.lang.NoSuchMethodError:
'java.lang.Object com.google.protobuf.DescriptorProtos$MessageOptions.getExtension(com.google.protobuf.GeneratedMessage$GeneratedExtension)'
    at scalapb.compiler.DescriptorImplicits$ExtendedMessageDescriptor.messageOptions(DescriptorImplicits.scala:532)
    at scalapb.compiler.DescriptorImplicits$ExtendedMessageDescriptor.sealedOneOfExtendsCount(DescriptorImplicits.scala:578)
    at scalapb.compiler.ProtoValidation.validateMessage(ProtoValidation.scala:107)
    at scalapb.compiler.ProtoValidation.$anonfun$validateFile$2(ProtoValidation.scala:17)
    at scalapb.compiler.ProtoValidation.$anonfun$validateFile$2$adapted(ProtoValidation.scala:17)
    at scala.collection.Iterator.foreach(Iterator.scala:943)
    at scala.collection.Iterator.foreach$(Iterator.scala:943)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
    at scala.collection.IterableLike.foreach(IterableLike.scala:74)
    at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
    at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
    at scalapb.compiler.ProtoValidation.validateFile(ProtoValidation.scala:17)
    at scalapb.compiler.ProtoValidation.$anonfun$validateFiles$1(ProtoValidation.scala:10)
    at scalapb.compiler.ProtoValidation.$anonfun$validateFiles$1$adapted(ProtoValidation.scala:10)
    at scala.collection.immutable.Stream.foreach(Stream.scala:533)
    at scalapb.compiler.ProtoValidation.validateFiles(ProtoValidation.scala:10)
    at scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator$.handleCodeGeneratorRequest(ExtraProtobufGenerator.scala:65)
    at scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator$.run(ExtraProtobufGenerator.scala:47)
    at protocbridge.frontend.PluginFrontend$.$anonfun$runWithBytes$1(PluginFrontend.scala:51)
    at scala.util.Try$.apply(Try.scala:213)
    at protocbridge.frontend.PluginFrontend$.runWithBytes(PluginFrontend.scala:51)
    at protocbridge.frontend.PluginFrontend$.runWithInputStream(PluginFrontend.scala:121)
    at protocbridge.frontend.PosixPluginFrontend$.$anonfun$prepare$2(PosixPluginFrontend.scala:40)
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
    at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1$$anon$2.block(ExecutionContextImpl.scala:75)
    at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3118)
    at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1.blockOn(ExecutionContextImpl.scala:87)
    at scala.concurrent.package$.blocking(package.scala:146)
    at protocbridge.frontend.PosixPluginFrontend$.$anonfun$prepare$1(PosixPluginFrontend.scala:38)
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
    at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
    at scala.util.Success.$anonfun$map$1(Try.scala:255)
    at scala.util.Success.map(Try.scala:213)
    at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
    at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:42)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:74)
    at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

java.lang.RuntimeException: Exit with code 1
    at scala.sys.package$.error(package.scala:30)
    at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
    at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
    at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
    at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
    at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)

ERROR: .../test/proto/BUILD:165:14
  scala @//test/proto:standalone_proto_strip_import_prefix_package
  failed: (Exit 1): scalapb_worker failed:
  error executing ProtoScalaPBRule command
  (from target //test/proto:standalone_proto_strip_import_prefix_package)
  bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker
    ... (remaining 2 arguments skipped)
```

This is the broken line:

- https://github.com/scalapb/ScalaPB/blob/v0.11.17/compiler-plugin/src/main/scala/scalapb/compiler/DescriptorImplicits.scala#L532

```scala
def messageOptions: MessageOptions = {
  val localOptions = message.getOptions.getExtension[MessageOptions](Scalapb.message)
```
mbland added a commit to mbland/rules_scala that referenced this issue Oct 21, 2024
Part of bazelbuild#1482.

These helper macros fix various repository name related errors when
building under Bzlmod, while remaining backwards compatible with
`WORKSPACE`.

Without Bzlmod, these macros return original repo names. With Bzlmod
enabled, they avoid the problems described below.

I've prepared a change for bazelbuild/bazel-skylib containing these
macros with full unit tests. If the maintainers accept that change, we
can bump our bazel_skylib version to use the macros from there, and
remove the `bzlmod.bzl` file.

---

Also includes a couple of other minor touch-ups:

- Updated the `runtime_deps` attribute in `repositories()` to add the
  Scala version suffix, just like `deps`.

- Added a `fail()` message to `repositories()` to make it more clear
  which Scala version dictionary is missing an artifact.

- Removed unnecessary internal uses of the `@io_bazel_rules_scala` repo
  name, applying `Label()` where necessary.

- Updated the construction of `dep_providers` in
  `_default_dep_providers` to remove the repo name, reduce duplication,
  and make the upcoming toolchain update slightly cleaner.

---

Before this change, `repositories()` would originally emit `BUILD`
targets including canonical repo names:

```py
scala_import(
    name = "_main~scala_deps~io_bazel_rules_scala_scala_compiler",
    jars = ["scala-compiler-2.12.18.jar"],
)
```

resulting in errors like:

```txt
ERROR: .../_main~_repo_rules~io_bazel_rules_scala/scala/BUILD:
  no such target '@@_main~scala_deps~io_bazel_rules_scala_scala_compiler//:io_bazel_rules_scala_scala_compiler':
  target 'io_bazel_rules_scala_scala_compiler' not declared in package ''
  defined by .../_main~scala_deps~io_bazel_rules_scala_scala_compiler/BUILD
  and referenced by '@@_main~_repo_rules~io_bazel_rules_scala//scala:default_toolchain_scala_compile_classpath_provider'
```

---

Attaching resources from custom repos to targets under Bzlmod, like in
`scalarules.test.resources.ScalaLibResourcesFromExternalDepTest`, would
break with:

```txt
$ bazel test //test/src/main/scala/scalarules/test/resources:all

1) Scala library depending on resources from external resource-only
  jar::allow to load resources(scalarules.test.resources.ScalaLibResourcesFromExternalDepTest)
  java.lang.NullPointerException
    at scalarules.test.resources.ScalaLibResourcesFromExternalDepTest.get(ScalaLibResourcesFromExternalDepTest.scala:17)
    at scalarules.test.resources.ScalaLibResourcesFromExternalDepTest.$anonfun$new$3(ScalaLibResourcesFromExternalDepTest.scala:11)
    at scalarules.test.resources.ScalaLibResourcesFromExternalDepTest.$anonfun$new$2(ScalaLibResourcesFromExternalDepTest.scala:11)
```

`_update_external_target_path` in `resources.bzl` fixes this problem.

---

Fixes `test_strict_deps_filter_included_target` from
`test/shell/test_strict_dependency.sh` when run under Bzlmod.

The `dependency_tracking_strict_deps_patterns` attribute of
//test_expect_failure/missing_direct_deps/filtering:plus_one_strict_deps_filter_a_impl
contains patterns starting with `@//`. However, in `_phase_dependency()`
from `scala/private/phases/phase_dependency.bzl`, these patterns were
compared against a stringified Label. Under Bazel < 7.1.0, this works
for root target Labels. Under Bazel >= 7.1.0, this breaks for root
target Labels under Bzlmod, which start with `@@//`.

`adjust_main_repo_prefix` updates the patterns accordingly in
`_partition_patterns` from `scala_toolchain.bzl`.
`apparent_repo_label_string` makes `_phase_dependency()` more resilient
when comparing target Labels against filters containing external
apparent repo names.

---

Fixes the `alias` targets generated by `_jvm_import_external` from
`scala_maven_import_external.bzl` by setting the `target` to the correct
apparent repo name.

Added `apparent_repo_name(repository_ctx.name)` to
`_jvm_import_external` to avoid this familiar error when running
`dt_patches/test_dt_patches` tests:

```txt
$ bazel build //...

ERROR: .../external/_main~compiler_source_repos~scala_reflect/BUILD:
  no such target '@@_main~compiler_source_repos~scala_reflect//:scala_reflect':
  target 'scala_reflect' not declared in package '' defined by
  .../external/_main~compiler_source_repos~scala_reflect/BUILD

ERROR: .../dt_patches/test_dt_patches/BUILD:11:22:
  no such target '@@_main~compiler_source_repos~scala_reflect//:scala_reflect':
  target 'scala_reflect' not declared in package '' defined by
  .../external/_main~compiler_source_repos~scala_reflect/BUILD
  and referenced by '//:dt_scala_toolchain_scala_compile_classpath_provider'

ERROR: Analysis of target
  '//:dt_scala_toolchain_scala_compile_classpath_provider' failed;
  build aborted: Analysis failed
```

---

As for why we need these macros, we can't rely on hacking the specific
canonical repository name format:

> Repos generated by extensions have canonical names in the form of
> `module_repo_canonical_name+extension_name+repo_name`. Note that the
> canonical name format is not an API you should depend on — it's
> subject to change at any time.
>
> - https://bazel.build/external/extension#repository_names_and_visibility

The change to no longer encode module versions in canonical repo names in
Bazel 7.1.0 is a recent example of Bazel maintainers altering the format:

- bazelbuild/bazel#21316

And the maintainers recently replaced the `~` delimiter with `+` in the
upcoming Bazel 8 release due to build performance issues on Windows:

- bazelbuild/bazel#22865

The core `apparent_repo_name` function assumes the only valid repo name
characters are letters, numbers, '_', '-', and '.'. This is valid so
long as this condition holds:

- https://github.com/bazelbuild/bazel/blob/7.3.2/src/main/java/com/google/devtools/build/lib/cmdline/RepositoryName.java#L159-L162
mbland added a commit to mbland/rules_scala that referenced this issue Oct 21, 2024
Part of bazelbuild#1482.

These are mostly small changes to make test assertions more flexible
between `WORKSPACE` and Bzlmod runs. For Bzlmod runs:

- Fixed `test_scala_config_content` from `test_scala_config.sh` by
  changing a path from `external/io_bazel_rules_scala_config` to
  `external/*io_bazel_rules_scala_config`.

- Fixed a number of tests by updating expected output messages to allow
  them to start with either `@//` or `@@//`.

- Fixed `test_stamped_target_label_loading` from
  `test/shell/test_strict_dependency.sh` by accommodating the canonical
  `io_bazel_rules_scala_guava` repo name. Also allows for the optional
  current Scala version suffix.

Also made these other important changes:

- Updated all the assertions in `test_helper.sh` to use Bash builtin regex
  matching via `_expect_failure_with_messages` instead of `grep`. This
  allows the expected message patterns to use full regular expressions
  while avoiding forking a new process. This new function helped reduce
  duplication in that file at the same time.

- Added `--repo_env="SCALA_VERSION=..."` to each test script called from
  `./test_coverage.sh`, and set `SCALA_VERSION` to 2.12.19 in each of
  these files. Using other Scala versions technically works, but the
  output is slightly different, causing the `diff` commands in the test
  cases to fail.

- Updated `test_version.sh` to copy the top level `.bazelversion` file
  into its test repo.

- Changed how `test_version.sh` handles injecting `twitter_scrooge`
  repos into the `WORKSPACE` file. This will make it easy to do the
  equivalent for `MODULE.bazel` when the time comes.

Also includes a few minor, opportunistic formatting cleanups.
mbland added a commit to mbland/rules_scala that referenced this issue Oct 21, 2024
Part of bazelbuild#1482.

Bumps every Scala version up to use protobuf-java:4.28.2, to
preemptively avoid the following error produced by upcoming Scalafmt
updates:

```txt
$ bazel test --repo_env=SCALA_VERSION=2.11.12 //test/scalafmt/...

INFO: Analyzed 9 targets (80 packages loaded, 3307 targets configured).
ERROR: .../test/scalafmt/BUILD:43:20: ScalaFmt
  test/scalafmt/test/scalafmt/formatted/formatted-test.scala.fmt.output
  failed: Worker process did not return a WorkResponse:

---8<---8<--- Start of log, file at .../bazel-workers/worker-134-ScalaFmt.log ---8<---8<---
Exception in thread "main" java.lang.NoSuchMethodError:
  'boolean com.google.protobuf.GeneratedMessageV3.isStringEmpty(java.lang.Object)'
  [ ...snip... ]
```

This issue seemed to suggest a library bump may fix it, and it did:

- protocolbuffers/protobuf#9236

Fortunately, even though this is a major version bump from 3.10.0 to
4.28.2, there were no compatibility issues, per:

> Protobuf major version releases may also be backwards-compatible with
> the last release of the previous major version. See the release notice
> for more details.
>
> - https://github.com/protocolbuffers/protobuf/tree/main/java#compatibility-notice
mbland added a commit to mbland/rules_scala that referenced this issue Oct 21, 2024
Part of bazelbuild#1482.

Replaces `native.new_local_repository` in `scalafmt_default_config` with
the new `scalafmt_config` repo rule. Resolves an incompatibility between
Bazel 6.5.0 and 7.3.2, while creating a much smaller `@scalafmt_default`
repo. Also updates the `native.register_toolchains` call with a
stringified `Label` to remove the hardcoding of `@io_bazel_rules_scala`.

The problem is that under Bazel 7.3.2, calling `scalafmt_default_config`
from a module extension produces this error:

```txt
$ bazel test //test/scalafmt/...

ERROR: Traceback (most recent call last):
  File ".../scala/extensions/deps.bzl",
    line 218, column 32, in _scala_deps_impl
    scalafmt_default_config()
  File ".../scala/scalafmt/scalafmt_repositories.bzl",
    line 18, column 32, in scalafmt_default_config
    native.new_local_repository(

Error in new_local_repository: The native module can be accessed only
  from a BUILD thread. Wrap the function in a macro and call it from a
  BUILD file
```

Ostensibly the solution is to replace `native.new_local_repository`
with:

```py
load(
  "@bazel_tools//tools/build_defs/repo:local.bzl",
  "new_local_repository",
)
```

However, `local.bzl` isn't available in Bazel 6.5.0:

```txt
$ bazel test //test/scalafmt/...

ERROR: Error computing the main repository mapping:
  at .../scala/scalafmt/scalafmt_repositories.bzl:8:6:
cannot load '@bazel_tools//tools/build_defs/repo:local.bzl':
  no such file
```

The new `scalafmt_config` repository rule works under both Bazel 6.5.0
and 7.3.2. Also, it symlinks only the single Scalafmt config file,
instead of every top level file and directory in the project, as
`new_local_repository` did.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
10 participants