This past year I've been working on resholve to improve Nix packaging for shell projects, and it's recently landed in nixpkgs-unstable
. This is a first-look at how you can put it to use.
The Shell Way conflicts with The Nix Way. Shell scripts need their environment to supply dependencies that aren't formally declared, but Nix wants to avoid leaking package dependencies into the user's environment. To resolve the conflict The Nix Way, we need to: find the dependencies, declare them, and make references to them absolute.
Manual review and the substitute* functions are good enough for small/simple scripts, but I've found at least one dependency missing from the Nix expression for all of the substantive Shell programs I've checked so far.
I'll start with a contrived example--an executable shell script which sources its own library script and invokes some other commands. If you want to follow along, clone the source with
1git clone https://gist.github.com/9d5a815b3ac49728a4893b003ab872c0.git
First, let's look at our executable script:
funsies.bash
1#! /usr/local/bin/bash
2source library.bash
3neofetch
It's a simple script--it just sources library.bash
and invokes neofetch
. But we'll have to take a look at library.bash
to be sure we understand correctly.
library.bash
1neofetch(){
2 cowsay -n < <(command neofetch --stdout)
3}
Aha! There's a catch in our library: funsies.bash
will run the library's neofetch
function--not the executable itself. Naively replacing neofetch
with ${neofetch}/bin/neofetch
would change the script's behavior. The library also has a dependency on cowsay that we could've missed if we just skimmed the executable script.
This is where resholve comes in. Instead of stdenv.mkDerivation
, our Nix expression uses resholve.mkDerivation
to build the package and tell resholve:
- which scripts to resolve
- which interpreter (shebang) to use
- to look for missing commands in the cowsay and neofetch packages
default.nix
1{ pkgs ? import <nixpkgs> { } }:
2
3with pkgs;
4resholvePackage rec {
5 version = "unset";
6 pname = "funsies";
7 src = ./.;
8 installPhase = ''
9 install -Dv library.bash $out/library.bash
10 install -Dv funsies.bash $out/bin/funsies.bash
11 '';
12 solutions = {
13 # an arbitrary name that you'd use to override this solution
14 # (some packages need multiple conflicting solutions)
15 funsies = {
16 # $out-relative paths
17 scripts = [ "library.bash" "bin/funsies.bash" ]; # 1
18 interpreter = "${bash_5}/bin/sh"; # 2
19 inputs = [ neofetch cowsay ]; # 3
20 };
21 };
22}
That's all--we're ready to nix-build
it:
build.console
1bash-5.1 $ nix-build
2this derivation will be built:
3 /nix/store/cgzwhy4wk360kdlidlq9rrpxc03ar1l7-funsies-unset.drv
4building '/nix/store/cgzwhy4wk360kdlidlq9rrpxc03ar1l7-funsies-unset.drv'...
5unpacking sources
6unpacking source archive /nix/store/rqj87qyg55768h6pda28bqx2k5zjazkq-write_a_simple_shell_package_with_resholve
7source root is write_a_simple_shell_package_with_resholve
8patching sources
9configuring
10no configure script, doing nothing
11building
12no Makefile, doing nothing
13installing
14install: creating directory '/nix/store/y764dgi2nnmdgzhp99a2zinhfvl8dwrm-funsies-unset'
15'library.bash' -> '/nix/store/y764dgi2nnmdgzhp99a2zinhfvl8dwrm-funsies-unset/library.bash'
16install: creating directory '/nix/store/y764dgi2nnmdgzhp99a2zinhfvl8dwrm-funsies-unset/bin'
17'funsies.bash' -> '/nix/store/y764dgi2nnmdgzhp99a2zinhfvl8dwrm-funsies-unset/bin/funsies.bash'
18post-installation fixup
19/nix/store/y764dgi2nnmdgzhp99a2zinhfvl8dwrm-funsies-unset /private/tmp/nix-build-funsies-unset.drv-0/write_a_simple_shell_package_with_resholve
20Overwrote '/nix/store/y764dgi2nnmdgzhp99a2zinhfvl8dwrm-funsies-unset/bin/funsies.bash'
21Overwrote '/nix/store/y764dgi2nnmdgzhp99a2zinhfvl8dwrm-funsies-unset/library.bash'
22/private/tmp/nix-build-funsies-unset.drv-0/write_a_simple_shell_package_with_resholve
23strip is /nix/store/rih2pjsr7p9dwwa6j3kdvpw4gdka7ik6-cctools-binutils-darwin-949.0.1/bin/strip
24stripping (with command strip and flags -S) in /nix/store/y764dgi2nnmdgzhp99a2zinhfvl8dwrm-funsies-unset/bin
25patching script interpreter paths in /nix/store/y764dgi2nnmdgzhp99a2zinhfvl8dwrm-funsies-unset
26/nix/store/y764dgi2nnmdgzhp99a2zinhfvl8dwrm-funsies-unset
During the fixup phase, resholve.mkDerivation
invokes resholve
to overwrite our copies of funsies.bash
and library.bash
with external dependencies resolved to the appropriate store paths. Let's take a look at the rewritten copies in the result
symlink, starting with funsies.bash
:
result/bin/funsies.bash
1#!/nix/store/bh237kq8sdsgp8l7m9c846fc9klpi4jn-bash-5.1-p4/bin/sh
2source /nix/store/y764dgi2nnmdgzhp99a2zinhfvl8dwrm-funsies-unset/library.bash
3neofetch
4
5### resholve directives (auto-generated) ## format_version: 2
6# resholve: keep source:/nix/store/y764dgi2nnmdgzhp99a2zinhfvl8dwrm-funsies-unset/library.bash
7
Here you can see that resholve:
- prepended the interpreter we specified (
${bash_5}/bin/sh
)
- resolved the reference to
library.bash
to an absolute store path
- was smart enough to avoid resolving
neofetch
, because it actually just refers to the function defined in library.bash
(It also appended information about how it resolved this script. You don't need to understand these lines. Each time resholve encounters a script that sources another script, it also parses the sourced script. When it does, it uses this list of directives to understand how it originally resolved the script.)
Let's see what it did to the library:
result/library.bash
1#!/nix/store/bh237kq8sdsgp8l7m9c846fc9klpi4jn-bash-5.1-p4/bin/sh
2neofetch(){
3 /nix/store/f7yvzpilpymw8356xm477fz35kmrkhhr-cowsay-3.03+dfsg2/bin/cowsay -n < <(command /nix/store/rbvydki2qkfp4v0qxs0nqiniqwh3cxwq-neofetch-7.1.0/bin/neofetch --stdout)
4}
5
6### resholve directives (auto-generated) ## format_version: 2
7# resholve: keep /nix/store/f7yvzpilpymw8356xm477fz35kmrkhhr-cowsay-3.03+dfsg2/bin/cowsay
8# resholve: keep /nix/store/rbvydki2qkfp4v0qxs0nqiniqwh3cxwq-neofetch-7.1.0/bin/neofetch
9
Most of this should make sense from before, but I'll pull out how it changed the main invocation for reference:
1 cowsay -n < <(command neofetch --stdout) # source
2/nix/store/.../cowsay -n < <(command /nix/store/.../neofetch --stdout) # resolved
Finally, let's take it for a spin:
run.console
1bash-5.1 $ ./result/bin/funsies.bash
2 _____________________________________________________
3/ abathur@085df404 \
4| ---------------- |
5| OS: macOS Catalina 10.15.7 19H2 x86_64 |
6| Host: MacBookAir9,1 |
7| Kernel: 19.6.0 |
8| Uptime: 13 days, 2 hours, 26 mins |
9| Packages: 7 (brew), 375 (nix-system), 2 (nix-user) |
10| Shell: bash 4.4.23 |
11| Resolution: 2560x1600 |
12| DE: Aqua |
13| WM: Quartz Compositor |
14| WM Theme: Graphite (Dark) |
15| Terminal: perl |
16| CPU: Intel i5-1030NG7 (8) @ 1.10GHz |
17| GPU: Intel Iris Plus Graphics |
18| Memory: 10976MiB / 16384MiB |
19\ /
20 -----------------------------------------------------
21 \ ^__^
22 \ (oo)\_______
23 (__)\ )\/\
24 ||----w |
25 || ||