-
-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bundix doesn't like architecture-specific gems #71
Comments
I have the same issue. It's trying to download from a url that doesn't exist (missing This is the correct path:
It's trying to fetch this:
|
I found an open PR for this. |
In the meantime, for those who would like to use Sorbet with Nix and Linux, the following works: let
# Import Nixpkgs however you like, be it <nixpkgs> or with Niv
pkgs = ...;
rubyGems = pkgs.bundlerEnv {
name = "gems-for-my-project";
ruby = pkgs.ruby_3_1;
gemdir = ./path/to/gemfile/directory;
gemConfig = pkgs.defaultGemConfig // {
# Poor man's solution: `sorbet-static` has a specific version per system, and Bundix currently does not support that.
# Additionally, the `sorbet` executable needs ELF patching.
# If the Sorbet version ever changes (after re-running `bundle update` then `bundix`) then the SHA256 must be updated accordingly.
# The simplest way to do so is to leave it empty and look at the error message to get the right hash.
# See https://nixos.org/manual/nixpkgs/stable/#developing-with-ruby
# See https://github.com/nix-community/bundix/issues/71
sorbet-static = attrs:
# Bundix picks the Darwin version by default, so we only need to patch on Linux
if pkgs.stdenv.isLinux then {
# Append the OS-specific version suffix
version = attrs.version + "-x86_64-linux";
# Update the SHA256 accordingly
source = attrs.source // { sha256 = "sha256-nuvwK1pQSJ4EsYdkT42q5NsG4QlZZuYT9Zxzz6839rk="; };
# Post-fix the gem by fixing the interpreter with the correct Nix one (default interpreter is `/lib64/ld-linux-x86-64.so.2` which usually does not exist on Nix machines)
nativeBuildInputs = with pkgs; [ autoPatchelfHook ];
} else attrs;
};
};
in
pkgs.mkShell {
buildInputs = [
# Bundler must be before `rubyGems` because the `bundle` executable provided by `rubyGems` does not allow updating the `Gemfile.lock` files
bundler
rubyGems.wrappedRuby
rubyGems
bundix
];
} The SHA256 part is not ideal, but otherwise it seems to work just fine.
Also, I think the code above can be easily adapted to other gems than Sorbet with a similar architecture-specific issue. I hope this is helpful to someone! |
This issue has been mentioned on NixOS Discourse. There might be relevant details there: |
Thanks @sir4ur0n this helped me a lot to figure out how to use autoPatchelfHook on the sorbet gem. I vendored a forked version of bundix @jdelStrother's patches to generate separate gemsets as well as support ruby 3, and this is my current thing that is so far working great for linux and arm mac sorbet let
overlays = [
(self: super: {
ruby = pkgs.ruby_3_1;
})
];
pkgs = import nixpkgs { inherit overlays system; };
bundlerPlatform = if pkgs.ruby.system == utils.system.aarch64-darwin then "arm64-darwin" else pkgs.ruby.system;
rubyEnv = pkgs.bundlerEnv {
name = "ruby-env";
inherit (pkgs) ruby;
gemdir = ./.;
gemset = .nix/gemset + ".${bundlerPlatform}.nix";
gemConfig = pkgs.defaultGemConfig // {
sorbet-static = attrs:
if pkgs.stdenv.isLinux then {
nativeBuildInputs = with pkgs; [ autoPatchelfHook ];
} else attrs;
};
}; |
This commit is an attempt to make Bundix more "platform-aware." Every rubygem has a "platform" field which names the ruby-platform that it can run on. For the vast majority of gems, this will be `"ruby"`, indicating that it's a gem written entirely in ruby and should be able to run on any ruby interpreter; however, there are some gems that rely on native extensions that might only work on certain systems. The previous `gemset.nix` format is generated exactly from the contents of the inputted `Gemfile.lock`. So if the `Gemfile.lock` included a reference to `nokogiri (1.14.2)`, Bundix would render a `gemset.nix` that included the sha256 hash for the ruby-platform version of that gem. The trouble with this behaviour is that both RubyGems and Bundler, at install time, will want to install the platform-specific version of the gem most suitable for the host system. So even though the `Gemfile.lock` references `nokogiri (1.14.2)`, if your host system's platform were `x86_64-linux`, they will try to install `nokogiri (1.14.2-x86-linux)` instead. This leads to issues like [1] and [2]. The new format of `gemfile.nix` is: ```nix { dependencies = ["list" "of" "top-level" "dependencies"]; platforms = { ruby = { gem-name = { dependencies = ["list" "of" 'transitive" "dependencies"] groups = [ ... ]; version = "..."; source = { # attrs describing the git, path, or RubyGems source }; }; }; other-platform = { gem-name = { ... }; }; }; } ``` The gemset's `dependencies` entry copies the `DEPENDENCIES` section of the `Gemfile.lock` and names all the top-level gem dependencies from the `Gemfile`. The gemset's `platforms` entry is an attrset where every key is a ruby-platform and its value is an attrset of the gems particular to that platform. This last attrset is essentially the same as the previous `gemset.nix` format. This commit also introduces a new nix function: `bundixEnv` (like `bundlerEnv`, but for Bundix). This function accepts the same arguments as `bundlerEnv`, with the addition of a `platform`. `bundixEnv` then converts the given gemset into a format suitable for `bundlerEnv` by selecting the gems appropriate for the given `platform`. Finally, it delegates to `bundlerEnv`, with the platform-specific gemset. `bundix --init` will generate an example `flake.nix` with an example package that demonstrates how `bundixEnv` works. See `gem help platforms` for more info about ruby platforms. The Bundler Guide [3] provides a few examples of rubygems that support multiple platforms. [1] nix-community#71 [2] https://discourse.nixos.org/t/issues-with-nix-reproducibility-on-macos-trying-to-build-nokogiri-ruby-error-unknown-warning-option/22019 [3] https://guides.rubygems.org/gems-with-extensions/
This commit is an attempt to make Bundix more "platform-aware." Every rubygem has a "platform" field which names the ruby-platform that it can run on. For the vast majority of gems, this will be `"ruby"`, indicating that it's a gem written entirely in ruby and should be able to run on any ruby interpreter; however, there are some gems that rely on native extensions that might only work on certain systems. The previous `gemset.nix` format only supported one version of each dependency. If the `Gemfile.lock` included a reference to `nokogiri (1.14.2)` and also `nokogiri (1.14.2-x86-linux)`, Bundix would render a `gemset.nix` that included only one of them. The trouble with this behaviour is that, at runtime, Bundler, will see has multiple nokogiri gems to choose from and may choose that one that was never built. This leads to issues like [1] and [2]. The new format of `gemfile.nix` is: ```nix { dependencies = ["list" "of" "top-level" "dependencies"]; platforms = { ruby = { gem-name = { dependencies = ["list" "of" 'transitive" "dependencies"] groups = [ ... ]; version = "..."; source = { # attrs describing the git, path, or RubyGems source }; }; }; other-platform = { gem-name = { ... }; }; }; } ``` The gemset's `dependencies` entry copies the `DEPENDENCIES` section of the `Gemfile.lock` and names all the top-level gem dependencies from the `Gemfile`. The gemset's `platforms` entry is an attrset where every key is a ruby-platform and its value is an attrset of the gems particular to that platform. This last attrset is essentially the same as the previous `gemset.nix` format. This commit also introduces a new nix function: `bundixEnv` (like `bundlerEnv`, but for Bundix). This function accepts the same arguments as `bundlerEnv`, with the addition of a `platform`. `bundixEnv` then converts the given gemset into a format suitable for `bundlerEnv` by selecting the gems appropriate for the given `platform`. Finally, it delegates to `bundlerEnv`, with the platform-specific gemset. `bundix --init` will generate an example `flake.nix` with an example package that demonstrates how `bundixEnv` works. See `gem help platforms` for more info about ruby platforms. The Bundler Guide [3] provides a few examples of rubygems that support multiple platforms. This commit also adds two guides, `guides/getting-started.md` and `guides/motivations.md` that describe this format in more detail. [1] nix-community#71 [2] https://discourse.nixos.org/t/issues-with-nix-reproducibility-on-macos-trying-to-build-nokogiri-ruby-error-unknown-warning-option/22019 [3] https://guides.rubygems.org/gems-with-extensions/
This commit is an attempt to make Bundix more "platform-aware." Every rubygem has a "platform" field which names the ruby-platform that it can run on. For the vast majority of gems, this will be `"ruby"`, indicating that it's a gem written entirely in ruby and should be able to run on any ruby interpreter; however, there are some gems that rely on native extensions that might only work on certain systems. The previous `gemset.nix` format only supported one version of each dependency. If the `Gemfile.lock` included a reference to `nokogiri (1.14.2)` and also `nokogiri (1.14.2-x86-linux)`, Bundix would render a `gemset.nix` that included only one of them. The trouble with this behaviour is that, at runtime, Bundler, will see has multiple nokogiri gems to choose from and may choose that one that was never built. This leads to issues like [1] and [2]. The new format of `gemfile.nix` is: ```nix { dependencies = ["list" "of" "top-level" "dependencies"]; platforms = { ruby = { gem-name = { dependencies = ["list" "of" 'transitive" "dependencies"] groups = [ ... ]; version = "..."; source = { # attrs describing the git, path, or RubyGems source }; }; }; other-platform = { gem-name = { ... }; }; }; } ``` The gemset's `dependencies` entry copies the `DEPENDENCIES` section of the `Gemfile.lock` and names all the top-level gem dependencies from the `Gemfile`. The gemset's `platforms` entry is an attrset where every key is a ruby-platform and its value is an attrset of the gems particular to that platform. This last attrset is essentially the same as the previous `gemset.nix` format. This commit also introduces a new nix function: `bundixEnv` (like `bundlerEnv`, but for Bundix). This function accepts the same arguments as `bundlerEnv`, with the addition of a `platform`. `bundixEnv` then converts the given gemset into a format suitable for `bundlerEnv` by selecting the gems appropriate for the given `platform`. Finally, it delegates to `bundlerEnv`, with the platform-specific gemset. `bundix --init` will generate an example `flake.nix` with an example package that demonstrates how `bundixEnv` works. See `gem help platforms` for more info about ruby platforms. The Bundler Guide [3] provides a few examples of rubygems that support multiple platforms. This commit also adds two guides, `guides/getting-started.md` and `guides/motivations.md` that describe this format in more detail. [1] nix-community#71 [2] https://discourse.nixos.org/t/issues-with-nix-reproducibility-on-macos-trying-to-build-nokogiri-ruby-error-unknown-warning-option/22019 [3] https://guides.rubygems.org/gems-with-extensions/
This commit is an attempt to make Bundix more "platform-aware." Every rubygem has a "platform" field which names the ruby-platform that it can run on. For the vast majority of gems, this will be `"ruby"`, indicating that it's a gem written entirely in ruby and should be able to run on any ruby interpreter; however, there are some gems that rely on native extensions that might only work on certain systems. The previous `gemset.nix` format only supported one version of each dependency. If the `Gemfile.lock` included a reference to `nokogiri (1.14.2)` and also `nokogiri (1.14.2-x86-linux)`, Bundix would render a `gemset.nix` that included only one of them. The trouble with this behaviour is that, at runtime, Bundler, will see has multiple nokogiri gems to choose from and may choose that one that was never built. This leads to issues like [1] and [2]. The new format of `gemfile.nix` is: ```nix { dependencies = ["list" "of" "top-level" "dependencies"]; platforms = { ruby = { gem-name = { dependencies = ["list" "of" 'transitive" "dependencies"] groups = [ ... ]; version = "..."; source = { # attrs describing the git, path, or RubyGems source }; }; }; other-platform = { gem-name = { ... }; }; }; } ``` The gemset's `dependencies` entry copies the `DEPENDENCIES` section of the `Gemfile.lock` and names all the top-level gem dependencies from the `Gemfile`. The gemset's `platforms` entry is an attrset where every key is a ruby-platform and its value is an attrset of the gems particular to that platform. This last attrset is essentially the same as the previous `gemset.nix` format. This commit also introduces a new nix function: `bundixEnv` (like `bundlerEnv`, but for Bundix). This function accepts the same arguments as `bundlerEnv`, with the addition of a `platform`. `bundixEnv` then converts the given gemset into a format suitable for `bundlerEnv` by selecting the gems appropriate for the given `platform`. Finally, it delegates to `bundlerEnv`, with the platform-specific gemset. `bundix --init` will generate an example `flake.nix` with an example package that demonstrates how `bundixEnv` works. See `gem help platforms` for more info about ruby platforms. The Bundler Guide [3] provides a few examples of rubygems that support multiple platforms. This commit also adds two guides, `guides/getting-started.md` and `guides/motivations.md` that describe this format in more detail. [1] nix-community#71 [2] https://discourse.nixos.org/t/issues-with-nix-reproducibility-on-macos-trying-to-build-nokogiri-ruby-error-unknown-warning-option/22019 [3] https://guides.rubygems.org/gems-with-extensions/
I am trying to bundix the sorbet typechecker.
Gemfile
:default.nix
(I usecallPackage
on this):Because of #70 , I am trying to generate
gemset.nix
usingbundle update && bundix
. Here is what happens:First, bundler correctly downloads the gem that matches my arch (
x86_64-linux
):Then bundix downloads a Darwin gem when constructing
gemset.nix
?Attempting to build this fails:
Note that it's trying to download a gem file with no architecture. If you visit the gem page for
sorbet-static
, you'll see a download link that includes the arch. This seems to be getting lost somewhere. Not sure if that is abundlerApp
bug in nix code, or a problem with the generatedgemset.nix
. But I'm pretty sure that bundix looking at the Darwin gem when constructinggemset.nix
is wrong.The text was updated successfully, but these errors were encountered: