Skip to content

Commit

Permalink
bundlerEnv: Add support for precompiled 'nativeSources' gems
Browse files Browse the repository at this point in the history 
See also https://github.com/jdelStrother/bundix/tree/native-sources
  • Loading branch information
@jdelStrother
jdelStrother committed Apr 17, 2021
1 parent 0325fda commit e7316af
Showing 1 changed file with 15 additions and 7 deletions.
22 changes: 15 additions & 7 deletions pkgs/development/ruby-modules/bundled-common/functions.nix
View file
Original file line number Diff line number Diff line change
Expand Up @@ -83,11 +83,19 @@ in rec {
};
in res;

composeGemAttrs = ruby: gems: name: attrs: ((removeAttrs attrs ["platforms"]) // {
inherit ruby;
inherit (attrs.source) type;
source = removeAttrs attrs.source ["type"];
gemName = name;
gemPath = map (gemName: gems.${gemName}) (attrs.dependencies or []);
});
composeGemAttrs = ruby: gems: name: attrs:
let matchingSource = lib.findFirst (p:
if lib.hasPrefix "arm64-darwin" p.platform then ruby.stdenv.hostPlatform.system == "aarch64-darwin"
else if lib.hasPrefix "x86_64-darwin" p.platform then ruby.stdenv.hostPlatform.system == "x86_64-darwin"
else if p.platform == "x86_64-linux" then ruby.stdenv.hostPlatform.system == "x86_64-linux"
else false
) attrs.source (attrs.nativeSources or []);
in ((removeAttrs attrs ["platforms" "nativeSources"]) // {
inherit ruby;
inherit (matchingSource) type;
source = removeAttrs matchingSource ["type" "platform"];
gemName = name;
gemPath = map (gemName: gems.${gemName}) (attrs.dependencies or []);
version = if (matchingSource.platform or "ruby") == "ruby" then attrs.version else "${attrs.version}-${matchingSource.platform}";
});
}

6 comments on commit e7316af

@will
Copy link

@will will commented on e7316af Nov 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdelStrother hey this looks like exactly what I want. I've managed to run your branch of bundix, but I'm not sure how to run a different version of bundlerEnv. Could you share some tips? I'm pretty new to nix in general, so I think it's just a matter of not knowing what I don’t know at this point. I'm using a flake for the project if that matters at all.

@jdelStrother
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@will It's a little while since I looked at this, we didn't end up using this approach. What I have instead is this hack to rewrite the Gemfile.lock to avoid platform-specific gems entirely, which has worked ok for our purposes:

pkgs.bundlerEnv {
  name = "gems-for-audioboom";
  gemdir = ../.;
  inherit ruby;
  # bundlerEnv doesn't play well with bundler's multiplatform support.
  # AFAICT it tries to load platform-specific gems that haven't been packaged.
  # Remove the platforms from Gemfile.lock and just use the 'ruby' platform.
  lockfile = pkgs.runCommand "platformless-gemfile" {} ''
    sed '/^PLATFORMS/,/^$/d' ${../Gemfile.lock} > $out
    echo -e "PLATFORMS\n  ruby" >> $out
  '';

Ruby definitely feels a bit under-maintained in the nix community... if you share some code you're struggling with I'll try & take another look.

@will
Copy link

@will will commented on e7316af Nov 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of the gems (sorbet) builds native stuff with haskell I think, which seems like it'd be annoying to get working if you had to build it each time vs downloading the prebuilt versions. So I think your approach here seems like it'd be great

But I'm just using the bundlerEnv from wherever that comes from, and I'm not sure how to use an alternate version that has this changed composeGemAttrs

Haven't gotten to anywhere fancy in the flake yet, I think this is all pretty standard:

{
  description = "flake";

  inputs = {
    nixpkgs.url = "nixpkgs";
    utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, nixpkgs, utils }:
    utils.lib.eachDefaultSystem (system:
      let
        overlays = [
          (self: super: {
            ruby = pkgs.ruby_3_1;
          })
        ];
        pkgs = import nixpkgs { inherit overlays system; };
        rubyEnv = pkgs.bundlerEnv {
          name = "ruby-env";
          inherit (pkgs) ruby;
          gemdir = ./.;
        };
      in
      {
        devShell = pkgs.mkShell {
          packages = [
            pkgs.postgresql_13
            pkgs.watchman # for sorbet lsp
          ];

          buildInputs = [
            rubyEnv
            rubyEnv.wrappedRuby
          ];
        };
      }
    );
}

When you were developing that alternate version of composeGemAttrs how did you iterate on it?

@jdelStrother
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Urgh, yeah - someone was asking about this with sorbet a while back (nix-community/bundix#71)

When you were developing that alternate version of composeGemAttrs how did you iterate on it?

IIRC I didn't find a good way to patch it into nixpkgs (eg as an overlay or whatever), so would just have nixpkgs checked out locally and in my shell.nix have something like pkgs = import ../vendor/nixpkgs {}.
Unless I'm missing something it's a little more cumbersome with flakes - you can set, eg, nixpkgs.url = "/Users/jon/Developer/vendor/nixpkgs";, but AFAIK any changes you make in your local nixpkgs directory need to be git-committed and then you'd need nix flake update each time you make a change.

I did briefly try my multiplatform branches with sorbet - sadly they don't magically fix it, looks like there's still some work to be done there 😞

@will
Copy link

@will will commented on e7316af Nov 26, 2022 via email •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@inscapist
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit late to the scene! @jdelStrother I just published a flake that made use of your work here. It supports native sources, among other things.

https://github.com/sagittaros/ruby-nix

Please sign in to comment.