advanced shell packaging: resholve YADM's nixpkg

I've been working on resholve to improve Nix packaging for shell projects since early 2020, but the new v0.6.0 release is a decent leap forward in the fraction of Shell scripts and programs resholve can handle. There's a more basic introduction to using resholve in my previous post (write a simple shell package with resholve), but some readers were interested in seeing a bit more detail on using resholve. As I'm pulling the release together and waiting for some feedback on release candidates, I took a little time to record a screencast where I walk through using resholve to re-package a complex Shell project.

The video is about 36 minutes long, though, so I also wanted to summarize it for anyone who is curious, but not 36-minutes curious. :)

If you want to follow along with the post, you can clone the source with:

1git clone https://gist.github.com/75978730de1715570e145a8073b4d651.git

Note: Nix sources in the video and post slightly differ. In the video I work directly on the nixpkgs source, but the post is using standalone Nix expressions.

Background

resholve is a command-line tool that helps you find and resolve external dependencies in Shell scripts.

The main reason you'd want to do this is to make Shell easier to package or deploy, but the need for another tool may not be evident if you don't spend much time dealing with Shell.

The flexibility that makes Shell a good glue language also makes it tricky to package.

Without a build process to gripe at you it's even hard to know whether you have identified all of the external dependencies of a complex Shell program --especially one that SOURCES a lot of other files and uses a lot of functions.

Even if a dependency is installed the user's PATH may not include the right directories, or they may be in the wrong order, or the script's author may have hard-coded absolute paths that aren't right for your OS or package manager.

YADM's existing Nix Package

Before I jump in to converting the package I want to show, succinctly, what's wrong with YADM's existing package by trying to run yadm in a pure nix shell (which only includes yadm, and Nix's set of standard build tools) on the PATH.

missing_git.console
1bash-5.1 $ nix-shell --pure -p yadm --run yadm
2/nix/store/7agjh6y20b2pn17sp9hrlx07f33y23wm-yadm-3.1.0/bin/yadm: line 1650: git: command not found
3/nix/store/7agjh6y20b2pn17sp9hrlx07f33y23wm-yadm-3.1.0/bin/yadm: line 857: git: command not found
4ERROR: This functionality requires Git to be installed, but the command 'git' cannot be located.

YADM is a git-based dotfile manager, so the invocation breaks because it expects to find git via the PATH and I didn't include the git package.

Here's the source for the existing package:

yadm_before.nix
 1{ pkgs ? import <nixpkgs> { } }:
 2
 3with pkgs;
 4stdenv.mkDerivation rec {
 5  pname = "yadm";
 6  version = "3.1.0";
 7
 8  buildInputs = [ git gnupg ];
 9
10  nativeBuildInputs = [ installShellFiles ];
11
12  src = fetchFromGitHub {
13    owner  = "TheLocehiliosan";
14    repo   = "yadm";
15    rev    = version;
16    sha256 = "0ga0p28nvqilswa07bzi93adk7wx6d5pgxlacr9wl9v1h6cds92s";
17  };
18
19  dontConfigure = true;
20  dontBuild = true;
21
22  installPhase = ''
23    runHook preInstall
24    install -Dt $out/bin yadm
25    runHook postInstall
26  '';
27
28  postInstall = ''
29    installManPage yadm.1
30    installShellCompletion --cmd yadm \
31      --zsh completion/zsh/_yadm \
32      --bash completion/bash/yadm
33  '';
34
35  meta = {
36    homepage = "https://github.com/TheLocehiliosan/yadm";
37    description = "Yet Another Dotfiles Manager";
38    longDescription = ''
39      yadm is a dotfile management tool with 3 main features:
40      * Manages files across systems using a single Git repository.
41      * Provides a way to use alternate files on a specific OS or host.
42      * Supplies a method of encrypting confidential data so it can safely be stored in your repository.
43    '';
44    license = lib.licenses.gpl3Plus;
45    maintainers = with lib.maintainers; [ abathur ];
46    platforms = lib.platforms.unix;
47  };
48}
49

The highlighted line shows that git is in the package's buildInputs, but buildInputs don't get pulled into the PATH with the package. Only the package itself is added to the PATH.

This is fine for users whose PATH already contains a compatible version of git. But, as with Shell scripts and programs everywhere yadm's Nix package would be happy to misfire if you (or a script, or a daemon) used it with the wrong environment.

The Nix ecosystem already has a few ways to address this kind of problem, including patching Shell wrappers and the propagatedUserEnvPkgs attribute. Each has its own problems but they share a fatal flaw: all of them require you to know the script's dependencies.

Conversion process

resholve's goal is to spot the most-common issues here, give you a toolkit for triaging them, and exit with an error until you do.

In this section I'll zoom in on a few of the most-important conversion steps, but I won't discuss every last step as the video does.

Starting point

Here's the source after the initial conversion to use the resholve API. This is, most-succinctly, the part where I actually convert it to use resholve:

yadm_resholving_1.nix
 1{ pkgs ? import <nixpkgs> { } }:
 2
 3with pkgs;
 4resholvePackage rec {
 5  pname = "yadm";
 6  version = "3.1.0";
 7
 8  nativeBuildInputs = [ installShellFiles ];
 9
10  src = fetchFromGitHub {
11    owner  = "TheLocehiliosan";
12    repo   = "yadm";
13    rev    = version;
14    sha256 = "0ga0p28nvqilswa07bzi93adk7wx6d5pgxlacr9wl9v1h6cds92s";
15  };
16
17  dontConfigure = true;
18  dontBuild = true;
19
20  installPhase = ''
21    runHook preInstall
22    install -Dt $out/bin yadm
23    runHook postInstall
24  '';
25
26  postInstall = ''
27    installManPage yadm.1
28    installShellCompletion --cmd yadm \
29      --zsh completion/zsh/_yadm \
30      --bash completion/bash/yadm
31  '';
32
33  solutions = {
34    yadm = {
35      scripts = [ "bin/yadm" ];
36      interpreter = "${bash}/bin/sh";
37      inputs = [
38        git
39      ];
40    };
41  };
42
43  passthru.tests = {
44    minimal = runCommand "${pname}-test" {} ''
45      export HOME=$out
46      ${yadm}/bin/yadm init
47    '';
48  };
49
50  meta = {
51    homepage = "https://github.com/TheLocehiliosan/yadm";
52    description = "Yet Another Dotfiles Manager";
53    longDescription = ''
54      yadm is a dotfile management tool with 3 main features:
55      * Manages files across systems using a single Git repository.
56      * Provides a way to use alternate files on a specific OS or host.
57      * Supplies a method of encrypting confidential data so it can safely be stored in your repository.
58    '';
59    license = lib.licenses.gpl3Plus;
60    maintainers = with lib.maintainers; [ abathur ];
61    platforms = lib.platforms.unix;
62  };
63}

The rest of the conversion process is basically working in a build-understand-triage loop with resholve to tell it what to do about the things it identifies. I started off from ~scratch, including only the git dependency that I've already identified.

Here's what resholve kicks out:

build_resholving_1.console
 1bash-5.1 $ nix-build yadm_resholving_1.nix --out-link result_resholving_1
 2this derivation will be built:
 3  /nix/store/8gycx0gqm88b2g645z8v0cwl9ba4bwss-yadm-3.1.0.drv
 4building '/nix/store/8gycx0gqm88b2g645z8v0cwl9ba4bwss-yadm-3.1.0.drv'...
 5unpacking sources
 6unpacking source archive /nix/store/swy5n5gagcwygj7w1gjlaf5g1rbklll7-source
 7source root is source
 8patching sources
 9installing
10post-installation fixup
11[resholve context] : changing directory to /nix/store/12xg9d5y4qhb53hagnk7fr3fcv2hv6kg-yadm-3.1.0
12[resholve context] RESHOLVE_LORE=/nix/store/rs98ww426yzzkmln4bfqrimn792b55nn-more-binlore
13[resholve context] RESHOLVE_INPUTS=/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin
14[resholve context] RESHOLVE_INTERPRETER=/nix/store/pcjan45rssdn01cxx3sjg70avjg6c3ni-bash-4.4-p23/bin/sh
15[resholve context] resholve --overwrite bin/yadm
16      $YADM_COMMAND "${YADM_ARGS[@]}"
17      ^~~~~~~~~~~~~
18/nix/store/12xg9d5y4qhb53hagnk7fr3fcv2hv6kg-yadm-3.1.0/bin/yadm:147: Can't resolve dynamic command
19error: builder for '/nix/store/8gycx0gqm88b2g645z8v0cwl9ba4bwss-yadm-3.1.0.drv' failed with exit code 7

I updated resholvePackage to let us know how it's invoking resholve in case we need to manually debug anything.

Triaging dynamic commands

resholve is complaining because the script invokes whatever is in the variable YADM_COMMAND. Since the command may be dynamic resholve makes us triage.

To figure out what to do with it, I go take a look at how yadm uses this variable:

lines 80-163 of yadm
 80function main() {
 81
 82  require_git
 83
 84  # capture full command, for passing to hooks
 85  # the parameters will be space delimited and
 86  # spaces, tabs, and backslashes will be escaped
 87  _tab=$'\t'
 88  for param in "$@"; do
 89    param="${param//\\/\\\\}"
 90    param="${param//$_tab/\\$_tab}"
 91    param="${param// /\\ }"
 92    _fc+=( "$param" )
 93  done
 94  FULL_COMMAND="${_fc[*]}"
 95
 96  # create the YADM_DIR & YADM_DATA if they doesn't exist yet
 97  [ -d "$YADM_DIR" ]  || mkdir -p "$YADM_DIR"
 98  [ -d "$YADM_DATA" ] || mkdir -p "$YADM_DATA"
 99
100  # parse command line arguments
101  local retval=0
102  internal_commands="^(alt|bootstrap|clean|clone|config|decrypt|encrypt|enter|git-crypt|help|--help|init|introspect|list|perms|transcrypt|upgrade|version|--version)$"
103  if [ -z "$*" ] ; then
104    # no argumnts will result in help()
105    help
106  elif [[ "$1" =~ $internal_commands ]] ; then
107    # for internal commands, process all of the arguments
108    YADM_COMMAND="${1//-/_}"
109    YADM_COMMAND="${YADM_COMMAND/__/}"
110    YADM_ARGS=()
111    shift
112
113    # commands listed below do not process any of the parameters
114    if [[ "$YADM_COMMAND" =~ ^(enter|git_crypt)$ ]] ; then
115      YADM_ARGS=("$@")
116    else
117      while [[ $# -gt 0 ]] ; do
118        key="$1"
119        case $key in
120          -a) # used by list()
121            LIST_ALL="YES"
122          ;;
123          -d) # used by all commands
124            DEBUG="YES"
125          ;;
126          -f) # used by init(), clone() and upgrade()
127            FORCE="YES"
128          ;;
129          -l) # used by decrypt()
130            DO_LIST="YES"
131            [[ "$YADM_COMMAND" =~ ^(clone|config)$ ]] && YADM_ARGS+=("$1")
132          ;;
133          -w) # used by init() and clone()
134            YADM_WORK="$(qualify_path "$2" "work tree")"
135            shift
136          ;;
137          *) # any unhandled arguments
138            YADM_ARGS+=("$1")
139          ;;
140        esac
141        shift
142      done
143    fi
144    [ ! -d "$YADM_WORK" ] && error_out "Work tree does not exist: [$YADM_WORK]"
145    HOOK_COMMAND="$YADM_COMMAND"
146    invoke_hook "pre"
147    $YADM_COMMAND "${YADM_ARGS[@]}"
148  else
149    # any other commands are simply passed through to git
150    HOOK_COMMAND="$1"
151    invoke_hook "pre"
152    git_command "$@"
153    retval="$?"
154  fi
155
156  # process automatic events
157  auto_alt
158  auto_perms
159  auto_bootstrap
160
161  exit_with_hook $retval
162
163}

The comment above the variable definition tells us this is for internal commands and the comment is backed up by the fact that it's set from the value of parameter 1 in the main function.

Since the value comes from the user at runtime the best way to triage this invocation is it keep it by adding a keep directive to the solution:

1keep = {
2  "$YADM_COMMAND" = true;
3};

Triaging variables that contain static commands

build_resholving_2.console
 1bash-5.1 $ nix-build yadm_resholving_2.nix --out-link result_resholving_2
 2this derivation will be built:
 3  /nix/store/7j28v1zf2k2677b262bsa1z1am2hs23h-yadm-3.1.0.drv
 4building '/nix/store/7j28v1zf2k2677b262bsa1z1am2hs23h-yadm-3.1.0.drv'...
 5unpacking sources
 6unpacking source archive /nix/store/swy5n5gagcwygj7w1gjlaf5g1rbklll7-source
 7source root is source
 8patching sources
 9installing
10post-installation fixup
11[resholve context] : changing directory to /nix/store/5c04gzx1hhwx7pmqjmf7s7kf72a8l473-yadm-3.1.0
12[resholve context] RESHOLVE_LORE=/nix/store/rs98ww426yzzkmln4bfqrimn792b55nn-more-binlore
13[resholve context] RESHOLVE_INPUTS=/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin
14[resholve context] RESHOLVE_INTERPRETER=/nix/store/pcjan45rssdn01cxx3sjg70avjg6c3ni-bash-4.4-p23/bin/sh
15[resholve context] RESHOLVE_KEEP='$YADM_COMMAND'
16[resholve context] resholve --overwrite bin/yadm
17    "${AWK_PROGRAM[0]}" \
18    ^
19/nix/store/5c04gzx1hhwx7pmqjmf7s7kf72a8l473-yadm-3.1.0/bin/yadm:425: Can't resolve dynamic command
20error: builder for '/nix/store/7j28v1zf2k2677b262bsa1z1am2hs23h-yadm-3.1.0.drv' failed with exit code 7

After rebuilding, resholve spits out the same basic error--though something named AWK_PROGRAM probably contains an external command. Once again, I jump into the yadm source to see how it's being used. Here's its declaration, alongside some similar ones:

lines 47-56 of yadm
47GPG_PROGRAM="gpg"
48OPENSSL_PROGRAM="openssl"
49GIT_PROGRAM="git"
50AWK_PROGRAM=("gawk" "awk")
51GIT_CRYPT_PROGRAM="git-crypt"
52TRANSCRYPT_PROGRAM="transcrypt"
53J2CLI_PROGRAM="j2"
54ENVTPL_PROGRAM="envtpl"
55ESH_PROGRAM="esh"
56LSB_RELEASE_PROGRAM="lsb_release"

Here's the actual invocation using it:

lines 425-434 of yadm
425  "${AWK_PROGRAM[0]}" \
426    -v class="$local_class" \
427    -v os="$local_system" \
428    -v host="$local_host" \
429    -v user="$local_user" \
430    -v distro="$local_distro" \
431    -v source="$input" \
432    -v source_dir="$(dirname "$input")" \
433    "$awk_pgm" \
434    "$input" > "$temp_file" || rm -f "$temp_file"

And here's a ~setup function that also depends on the variable:

lines 1704-1709 of yadm
1704function set_awk() {
1705  local pgm
1706  for pgm in "${AWK_PROGRAM[@]}"; do
1707    command -v "$pgm" &> /dev/null && AWK_PROGRAM=("$pgm") && return
1708  done
1709}

The ability to fix this practice of storing commands in variables is one of the new features in this release. Narrowly, I can fix resholve's complaint by adding gawk to this solution's inputs, along with a fix directive like:

1fix = {
2  "$AWK_PROGRAM" = [ "awk" ];
3}

This fix directive tells resholve to replace parameter expansions of AWK_PROGRAM with awk. It will replace the entire expansion (even if it's a complex braced one) before it resolves commands. Instances that are commands will get resolved later, and others will keep the bare replacement. This tries to strike a good balance between getting the commands resolved without having to worry about what else the script does with the variable.

As you saw in the yadm source, awk is only one of several commands that get this treatment. For the next step, I'll go ahead and add all of these, plus all of the remaining ~dynamic command instances like YADM_COMMAND.

yadm_resholving_2-3.diff
 1bash-5.1 $ git diff --no-index yadm_resholving_2.nix yadm_resholving_3.nix
 2diff --git a/yadm_resholving_2.nix b/yadm_resholving_3.nix
 3index 90fc523..5d8f5b2 100644
 4--- a/yadm_resholving_2.nix
 5+++ b/yadm_resholving_3.nix
 6@@ -36,9 +36,47 @@ resholvePackage rec {
 7       interpreter = "${bash}/bin/sh";
 8       inputs = [
 9         git
10+        gnupg
11+        openssl
12+        gawk
13+        /*
14+        TODO: yadm can use git-crypt and transcrypt
15+        but it does so in a way that resholve 0.6.0
16+        can't yet do anything smart about. It looks
17+        like these are for interactive use, so the
18+        main impact should just be that users still
19+        need both of these packages in their profile
20+        to support their use in yadm.
21+        */
22+        # git-crypt
23+        # transcrypt
24+        j2cli
25+        esh
26       ];
27+      fix = {
28+        "$GPG_PROGRAM" = [ "gpg" ];
29+        "$OPENSSL_PROGRAM" = [ "openssl" ];
30+        "$GIT_PROGRAM" = [ "git" ];
31+        "$AWK_PROGRAM" = [ "awk" ];
32+        # see inputs comment
33+        # "$GIT_CRYPT_PROGRAM" = [ "git-crypt" ];
34+        # "$TRANSCRYPT_PROGRAM" = [ "transcrypt" ];
35+        "$J2CLI_PROGRAM" = [ "j2" ];
36+        "$ESH_PROGRAM" = [ "esh" ];
37+        # not in nixpkgs
38+        # "$ENVTPL_PROGRAM" = [ "envtpl" ];
39+        # "$LSB_RELEASE_PROGRAM" = [ "lsb_release" ];
40+      };
41       keep = {
42-        "$YADM_COMMAND" = true;
43+        "$YADM_COMMAND" = true; # internal cmds
44+        "$template_cmd" = true; # dynamic, template-engine
45+        "$SHELL" = true; # probably user env? unsure
46+        "$hook_command" = true; # ~git hooks?
47+        "exec" = [ "$YADM_BOOTSTRAP" ]; # yadm bootstrap script
48+
49+        # not in nixpkgs
50+        "$ENVTPL_PROGRAM" = true;
51+        "$LSB_RELEASE_PROGRAM" = true;
52       };
53     };
54   };

Resolving nested commands

build_resholving_3.console
 1bash-5.1 $ nix-build yadm_resholving_3.nix --out-link result_resholving_3
 2this derivation will be built:
 3  /nix/store/4vrkfc5rqdgdnvlcynpljgw97im706yl-yadm-3.1.0.drv
 4building '/nix/store/4vrkfc5rqdgdnvlcynpljgw97im706yl-yadm-3.1.0.drv'...
 5unpacking sources
 6unpacking source archive /nix/store/swy5n5gagcwygj7w1gjlaf5g1rbklll7-source
 7source root is source
 8patching sources
 9installing
10post-installation fixup
11[resholve context] : changing directory to /nix/store/skghr1m65mh2bk47q7dv6f0f131fqfrq-yadm-3.1.0
12[resholve context] RESHOLVE_LORE=/nix/store/vimyjdi52n71xp4h8zmig5zks5dvz5y2-more-binlore
13[resholve context] RESHOLVE_FIX='$AWK_PROGRAM:awk $ESH_PROGRAM:esh $GIT_PROGRAM:git $GPG_PROGRAM:gpg $J2CLI_PROGRAM:j2 $OPENSSL_PROGRAM:openssl'
14[resholve context] RESHOLVE_INPUTS=/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin:/nix/store/y2hcv73jwa5k1s1dhj65mjz4cwyrfzm8-gnupg-2.2.27/bin:/nix/store/gdrl09cxsbml1gdb0dqkjb11i1s36183-openssl-1.1.1k-bin/bin:/nix/store/8kpxw5na07ggdl2bs8kiwysif7120r6g-gawk-5.1.0/bin:/nix/store/p0876isrv4gi1pbfkc6bms737grzwjlj-python3.8-j2cli-0.3.10/bin:/nix/store/irnmflhx17rlhzvz6g6d3f788aykf571-esh-0.1.1/bin
15[resholve context] RESHOLVE_INTERPRETER=/nix/store/pcjan45rssdn01cxx3sjg70avjg6c3ni-bash-4.4-p23/bin/sh
16[resholve context] RESHOLVE_KEEP='$ENVTPL_PROGRAM $LSB_RELEASE_PROGRAM $SHELL $YADM_COMMAND $hook_command $template_cmd exec:$YADM_BOOTSTRAP'
17[resholve context] resholve --overwrite bin/yadm
18    "esh" -o "$temp_file" "$input" \
19    ^
20/nix/store/skghr1m65mh2bk47q7dv6f0f131fqfrq-yadm-3.1.0/bin/yadm:476: 'esh' _might_ be able to execute its arguments, and I don't have any command-specific rules for figuring out if this specific invocation does or not. 
21error: builder for '/nix/store/4vrkfc5rqdgdnvlcynpljgw97im706yl-yadm-3.1.0.drv' failed with exit code 9

This error is part of another new feature. The update will add support for recursive resolution. This was easy enough for builtins, but an extra-tricky part of packaging shell is that any given external command might be able to execute other external commands.

sudo and xargs are the most-familiar examples of this-- but they are far from the only ones.

To address these, resholve now requires some metadata for each external command it finds. It outsources the job of generating this metadata, so it's passed in a format I call "lore". The most important kind of lore tells resholve whether an executable can, cannot, or might be able to execute its arguments.

resholve's Nix API automatically supplies lore for every package in the inputs using binlore, the reference lore provider I'm developing. This judgement is low-resolution. Binlore isn't doing advanced binary analysis. It's using some YARA rules to sniff for signs an executable uses an exec API.

When resholve finds invocations of executables that the lore indicates "can" or "might" execute their arguments, resholve will try to fall back on internal command-specific parsing rules for finding execed arguments. If it lacks parsing rules for the command, it throws the error we saw above:

'esh' might be able to execute its arguments, and I don't have any command-specific rules for figuring out if this specific invocation does or not.

It'll take some time (and I'll need some help) to refine these features in both binlore and resholve. But they're a starting point. For this demo I just triaged invocations of each potential execer manually. If they looked safe, I added a lore override to tell resholve to assume the executable is safe. I can override the execer lore for a single executable like this:

1execer = [
2  "cannot:${j2cli}/bin/j2"
3];

Caution: I'm cutting a corner! If a future version of the script started using exec behavior of an executable we override the lore for, resholve won't be able to help us.

The "best" way to address this is to look into whether binlore needs an override for this executable, or resholve needs an argument parser for it.

Once again, I added all of these at once.

Triaging simple missing commands

For completeness, I'll also demonstrate resholve's most-straightforward use case: resolving simple bare missing commands that can be supplied by an existing package.

build_resholving_4.console
 1bash-5.1 $ nix-build yadm_resholving_4.nix --out-link result_resholving_4
 2this derivation will be built:
 3  /nix/store/idm36d4c16l78f74v6adah3gmqz6zfyq-yadm-3.1.0.drv
 4building '/nix/store/idm36d4c16l78f74v6adah3gmqz6zfyq-yadm-3.1.0.drv'...
 5unpacking sources
 6unpacking source archive /nix/store/swy5n5gagcwygj7w1gjlaf5g1rbklll7-source
 7source root is source
 8patching sources
 9installing
10post-installation fixup
11[resholve context] : changing directory to /nix/store/hirchsqpqm3s8w0jcbgh83gs46qwanwk-yadm-3.1.0
12[resholve context] RESHOLVE_LORE=/nix/store/mxdwp8dxbh7wdn70c051wjl9yzx0nmnc-more-binlore
13[resholve context] RESHOLVE_EXECER='cannot:/nix/store/p0876isrv4gi1pbfkc6bms737grzwjlj-python3.8-j2cli-0.3.10/bin/j2 cannot:/nix/store/irnmflhx17rlhzvz6g6d3f788aykf571-esh-0.1.1/bin/esh cannot:/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git cannot:/nix/store/y2hcv73jwa5k1s1dhj65mjz4cwyrfzm8-gnupg-2.2.27/bin/gpg'
14[resholve context] RESHOLVE_FIX='$AWK_PROGRAM:awk $ESH_PROGRAM:esh $GIT_PROGRAM:git $GPG_PROGRAM:gpg $J2CLI_PROGRAM:j2 $OPENSSL_PROGRAM:openssl'
15[resholve context] RESHOLVE_INPUTS=/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin:/nix/store/y2hcv73jwa5k1s1dhj65mjz4cwyrfzm8-gnupg-2.2.27/bin:/nix/store/gdrl09cxsbml1gdb0dqkjb11i1s36183-openssl-1.1.1k-bin/bin:/nix/store/8kpxw5na07ggdl2bs8kiwysif7120r6g-gawk-5.1.0/bin:/nix/store/p0876isrv4gi1pbfkc6bms737grzwjlj-python3.8-j2cli-0.3.10/bin:/nix/store/irnmflhx17rlhzvz6g6d3f788aykf571-esh-0.1.1/bin:/nix/store/pcjan45rssdn01cxx3sjg70avjg6c3ni-bash-4.4-p23/bin:/nix/store/gh440dlrbabj33lj410w37qpjfidhgnd-gnutar-1.34/bin
16[resholve context] RESHOLVE_INTERPRETER=/nix/store/pcjan45rssdn01cxx3sjg70avjg6c3ni-bash-4.4-p23/bin/sh
17[resholve context] RESHOLVE_KEEP='$ENVTPL_PROGRAM $LSB_RELEASE_PROGRAM $SHELL $YADM_COMMAND $hook_command $template_cmd exec:$YADM_BOOTSTRAP'
18[resholve context] resholve --overwrite bin/yadm
19    [ -d "$YADM_DIR" ]  || mkdir -p "$YADM_DIR"
20                           ^~~~~
21/nix/store/hirchsqpqm3s8w0jcbgh83gs46qwanwk-yadm-3.1.0/bin/yadm:97: Couldn't resolve command 'mkdir'
22error: builder for '/nix/store/idm36d4c16l78f74v6adah3gmqz6zfyq-yadm-3.1.0.drv' failed with exit code 3

This output indicates that resholve found a bare invocation of mkdir, but none of the current inputs supply it. In this case, we can solve the error by adding coreutils to the inputs.

Triaging commands that can't be resolved

build_resholving_5.console
 1bash-5.1 $ nix-build yadm_resholving_5.nix --out-link result_resholving_5
 2this derivation will be built:
 3  /nix/store/n7m489dav4fi4pwpvsfk0xzk8745782y-yadm-3.1.0.drv
 4building '/nix/store/n7m489dav4fi4pwpvsfk0xzk8745782y-yadm-3.1.0.drv'...
 5unpacking sources
 6unpacking source archive /nix/store/swy5n5gagcwygj7w1gjlaf5g1rbklll7-source
 7source root is source
 8patching sources
 9installing
10post-installation fixup
11[resholve context] : changing directory to /nix/store/xhps4gff69lwdrmm0br3wcdmcw76lz1b-yadm-3.1.0
12[resholve context] RESHOLVE_LORE=/nix/store/1wsp3vhd1p1xjyf3fld01n0xs802v6si-more-binlore
13[resholve context] RESHOLVE_EXECER='cannot:/nix/store/p0876isrv4gi1pbfkc6bms737grzwjlj-python3.8-j2cli-0.3.10/bin/j2 cannot:/nix/store/irnmflhx17rlhzvz6g6d3f788aykf571-esh-0.1.1/bin/esh cannot:/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git cannot:/nix/store/y2hcv73jwa5k1s1dhj65mjz4cwyrfzm8-gnupg-2.2.27/bin/gpg'
14[resholve context] RESHOLVE_FIX='$AWK_PROGRAM:awk $ESH_PROGRAM:esh $GIT_PROGRAM:git $GPG_PROGRAM:gpg $J2CLI_PROGRAM:j2 $OPENSSL_PROGRAM:openssl'
15[resholve context] RESHOLVE_INPUTS=/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin:/nix/store/y2hcv73jwa5k1s1dhj65mjz4cwyrfzm8-gnupg-2.2.27/bin:/nix/store/gdrl09cxsbml1gdb0dqkjb11i1s36183-openssl-1.1.1k-bin/bin:/nix/store/8kpxw5na07ggdl2bs8kiwysif7120r6g-gawk-5.1.0/bin:/nix/store/p0876isrv4gi1pbfkc6bms737grzwjlj-python3.8-j2cli-0.3.10/bin:/nix/store/irnmflhx17rlhzvz6g6d3f788aykf571-esh-0.1.1/bin:/nix/store/pcjan45rssdn01cxx3sjg70avjg6c3ni-bash-4.4-p23/bin:/nix/store/gh440dlrbabj33lj410w37qpjfidhgnd-gnutar-1.34/bin:/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin
16[resholve context] RESHOLVE_INTERPRETER=/nix/store/pcjan45rssdn01cxx3sjg70avjg6c3ni-bash-4.4-p23/bin/sh
17[resholve context] RESHOLVE_KEEP='$ENVTPL_PROGRAM $LSB_RELEASE_PROGRAM $SHELL $YADM_COMMAND $hook_command $template_cmd exec:$YADM_BOOTSTRAP'
18[resholve context] resholve --overwrite bin/yadm
19      cygpath -u "$1"
20      ^~~~~~~
21/nix/store/xhps4gff69lwdrmm0br3wcdmcw76lz1b-yadm-3.1.0/bin/yadm:2139: Couldn't resolve command 'cygpath'
22error: builder for '/nix/store/n7m489dav4fi4pwpvsfk0xzk8745782y-yadm-3.1.0.drv' failed with exit code 3

As we just saw, the typical response when resholve reports a missing command is to just add a package that includes it to the inputs.

But this isn't always possible. In this case, cygpath is from cygwin. I don't think we have it in nixpkgs, so the best approach is to use a fake directive to tell resholve to pretend something is defined (when there's a really good reason it's missing).

Other examples of when this approach is useful:

  • Fake a function that a Shell library calls but does not define (because it is a hook/callback that sourcing scripts are supposed to fefine).
  • Fake a builtin for some other shell (or if you're using loadable builtins and such.)

In this case, I'll only fake cygpath when the stdenv is not cygwin:

1fake = {
2  external = if stdenv.isCygwin then [ ] else [ "cygpath" ];
3};

This will hopefully let cygpath break naturally if someone does try to build this package on cygwin, and then figure out the full / correct fix when that happens.

build_after.console
1bash-5.1 $ nix-build yadm_after.nix --out-link result_after
2/nix/store/g424p1fcphah2jmc1yq5hsv4rmp3nzyf-yadm-3.1.0

Result

If we get to the other side of the conversion process, the dependencies we were able to specify should be nailed down. For reference, here's a before/after diff illustrating what resholve did to this script:

resolved YADM
  1bash-5.1 $ git diff --no-index result_before/bin/yadm result_after/bin/yadm
  2diff --git a/result_before/bin/yadm b/result_after/bin/yadm
  3index 9f57f60..f8340d8 100755
  4--- a/result_before/bin/yadm
  5+++ b/result_after/bin/yadm
  6@@ -18,7 +18,7 @@
  7 # shellcheck shell=bash
  8 # execute script with bash (shebang line is /bin/sh for portability)
  9 if [ -z "$BASH_VERSION" ]; then
 10-  [ "$YADM_TEST" != 1 ] && exec bash "$0" "$@"
 11+  [ "$YADM_TEST" != 1 ] && exec /nix/store/pcjan45rssdn01cxx3sjg70avjg6c3ni-bash-4.4-p23/bin/bash "$0" "$@"
 12 fi
 13 
 14 VERSION=3.1.0
 15@@ -94,8 +94,8 @@ function main() {
 16   FULL_COMMAND="${_fc[*]}"
 17 
 18   # create the YADM_DIR & YADM_DATA if they doesn't exist yet
 19-  [ -d "$YADM_DIR" ]  || mkdir -p "$YADM_DIR"
 20-  [ -d "$YADM_DATA" ] || mkdir -p "$YADM_DATA"
 21+  [ -d "$YADM_DIR" ]  || /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mkdir -p "$YADM_DIR"
 22+  [ -d "$YADM_DATA" ] || /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mkdir -p "$YADM_DATA"
 23 
 24   # parse command line arguments
 25   local retval=0
 26@@ -422,16 +422,16 @@ function conditions() {
 27 }
 28 EOF
 29 
 30-  "${AWK_PROGRAM[0]}" \
 31+  "/nix/store/8kpxw5na07ggdl2bs8kiwysif7120r6g-gawk-5.1.0/bin/awk" \
 32     -v class="$local_class" \
 33     -v os="$local_system" \
 34     -v host="$local_host" \
 35     -v user="$local_user" \
 36     -v distro="$local_distro" \
 37     -v source="$input" \
 38-    -v source_dir="$(dirname "$input")" \
 39+    -v source_dir="$(/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/dirname "$input")" \
 40     "$awk_pgm" \
 41-    "$input" > "$temp_file" || rm -f "$temp_file"
 42+    "$input" > "$temp_file" || /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/rm -f "$temp_file"
 43 
 44   move_file "$input" "$output" "$temp_file"
 45 }
 46@@ -447,7 +447,7 @@ function template_j2cli() {
 47   YADM_USER="$local_user"     \
 48   YADM_DISTRO="$local_distro" \
 49   YADM_SOURCE="$input"        \
 50-  "$J2CLI_PROGRAM" "$input" -o "$temp_file"
 51+  "/nix/store/p0876isrv4gi1pbfkc6bms737grzwjlj-python3.8-j2cli-0.3.10/bin/j2" "$input" -o "$temp_file"
 52 
 53   move_file "$input" "$output" "$temp_file"
 54 }
 55@@ -473,7 +473,7 @@ function template_esh() {
 56   output="$2"
 57   temp_file="${output}.$$.$RANDOM"
 58 
 59-  "$ESH_PROGRAM" -o "$temp_file" "$input" \
 60+  "/nix/store/irnmflhx17rlhzvz6g6d3f788aykf571-esh-0.1.1/bin/esh" -o "$temp_file" "$input" \
 61   YADM_CLASS="$local_class"   \
 62   YADM_OS="$local_system"     \
 63   YADM_HOSTNAME="$local_host" \
 64@@ -494,9 +494,9 @@ function move_file() {
 65   # if the output files already exists as read-only, change it to be writable.
 66   # there are some environments in which a read-only file will prevent the move
 67   # from being successful.
 68-  [[ -e "$output" && ! -w "$output" ]] && chmod u+w "$output"
 69+  [[ -e "$output" && ! -w "$output" ]] && /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/chmod u+w "$output"
 70 
 71-  mv -f "$temp_file" "$output"
 72+  /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mv -f "$temp_file" "$output"
 73   copy_perms "$input" "$output"
 74 }
 75 
 76@@ -528,7 +528,7 @@ function alt() {
 77   # determine all tracked files
 78   local tracked_files=()
 79   local IFS=$'\n'
 80-  for tracked_file in $("$GIT_PROGRAM" ls-files | LC_ALL=C sort); do
 81+  for tracked_file in $("/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" ls-files | LC_ALL=C /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/sort); do
 82     tracked_files+=("$tracked_file")
 83   done
 84 
 85@@ -593,12 +593,12 @@ function remove_stale_links() {
 86   if readlink_available; then
 87     for stale_candidate in "${possible_alts[@]}"; do
 88       if [ -L "$stale_candidate" ]; then
 89-        src=$(readlink "$stale_candidate" 2>/dev/null)
 90+        src=$(/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/readlink "$stale_candidate" 2>/dev/null)
 91         if [ -n "$src" ]; then
 92           for review_link in "${alt_linked[@]}"; do
 93             [ "$src" = "$review_link" ] && continue 2
 94           done
 95-          rm -f "$stale_candidate"
 96+          /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/rm -f "$stale_candidate"
 97         fi
 98       fi
 99     done
100@@ -616,13 +616,13 @@ function set_local_alt_values() {
101 
102   local_host="$(config local.hostname)"
103   if [ -z "$local_host" ] ; then
104-    local_host=$(uname -n)
105+    local_host=$(/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/uname -n)
106     local_host=${local_host%%.*} # trim any domain from hostname
107   fi
108 
109   local_user="$(config local.user)"
110   if [ -z "$local_user" ] ; then
111-    local_user=$(id -u -n)
112+    local_user=$(/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/id -u -n)
113   fi
114 
115   local_distro="$(query_distro)"
116@@ -636,7 +636,7 @@ function alt_linking() {
117   local alt_sources=()
118   local alt_template_cmds=()
119 
120-  for alt_path in $(for tracked in "${tracked_files[@]}"; do printf "%s\n" "$tracked" "${tracked%/*}"; done | LC_ALL=C sort -u) "${ENCRYPT_INCLUDE_FILES[@]}"; do
121+  for alt_path in $(for tracked in "${tracked_files[@]}"; do printf "%s\n" "$tracked" "${tracked%/*}"; done | LC_ALL=C /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/sort -u) "${ENCRYPT_INCLUDE_FILES[@]}"; do
122     alt_path="$YADM_BASE/$alt_path"
123     if [[ "$alt_path" =~ .\#\#. ]]; then
124       if [ -e "$alt_path" ] ; then
125@@ -656,7 +656,7 @@ function alt_linking() {
126       # ensure the destination path exists
127       assert_parent "$tgt"
128       # remove any existing symlink before processing template
129-      [ -L "$tgt" ] && rm -f "$tgt"
130+      [ -L "$tgt" ] && /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/rm -f "$tgt"
131       "$template_cmd" "$src" "$tgt"
132     elif [ -n "$src" ]; then
133       # a link source is defined, create symlink
134@@ -666,8 +666,8 @@ function alt_linking() {
135       assert_parent "$tgt"
136       if [ "$do_copy" -eq 1 ]; then
137         # remove any existing symlink before copying
138-        [ -L "$tgt" ] && rm -f "$tgt"
139-        cp -f "$src" "$tgt"
140+        [ -L "$tgt" ] && /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/rm -f "$tgt"
141+        /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/cp -f "$src" "$tgt"
142       else
143         ln_relative "$src" "$tgt"
144       fi
145@@ -686,7 +686,7 @@ function ln_relative() {
146   fi
147   local rel_source
148   rel_source=$(relative_path "$target_dir" "$full_source")
149-  ln -nfs "$rel_source" "$full_target"
150+  /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/ln -nfs "$rel_source" "$full_target"
151   alt_linked+=("$rel_source")
152 }
153 
154@@ -750,8 +750,8 @@ function clone() {
155   # remove existing if forcing the clone to happen anyway
156   [ -d "$YADM_REPO" ] && {
157     debug "Removing existing repo prior to clone"
158-    "$GIT_PROGRAM" -C "$YADM_WORK" submodule deinit -f --all
159-    rm -rf "$YADM_REPO"
160+    "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" -C "$YADM_WORK" submodule deinit -f --all
161+    /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/rm -rf "$YADM_REPO"
162   }
163 
164   local wc
165@@ -761,22 +761,22 @@ function clone() {
166   # first clone without checkout
167   debug "Doing an initial clone of the repository"
168   (cd "$wc" &&
169-       "$GIT_PROGRAM" -c core.sharedrepository=0600 clone --no-checkout \
170+       "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" -c core.sharedrepository=0600 clone --no-checkout \
171                       --separate-git-dir="$YADM_REPO" "${args[@]}" repo.git) || {
172       debug "Removing repo after failed clone"
173-      rm -rf "$YADM_REPO" "$wc"
174+      /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/rm -rf "$YADM_REPO" "$wc"
175       error_out "Unable to clone the repository"
176   }
177-  rm -rf "$wc"
178+  /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/rm -rf "$wc"
179   configure_repo
180 
181   # then reset the index as the --no-checkout flag makes the index empty
182-  "$GIT_PROGRAM" reset --quiet -- .
183+  "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" reset --quiet -- .
184 
185   if [ "$YADM_WORK" = "$HOME" ]; then
186     debug "Determining if repo tracks private directories"
187     for private_dir in $(private_dirs all); do
188-      found_log=$("$GIT_PROGRAM" log -n 1 -- "$private_dir" 2>/dev/null)
189+      found_log=$("/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" log -n 1 -- "$private_dir" 2>/dev/null)
190       if [ -n "$found_log" ]; then
191         debug "Private directory $private_dir is tracked by repo"
192         assert_private_dirs "$private_dir"
193@@ -790,11 +790,11 @@ function clone() {
194 
195       cd_work "Clone" || return
196 
197-      "$GIT_PROGRAM" ls-files --deleted | while IFS= read -r file; do
198-          "$GIT_PROGRAM" checkout -- ":/$file"
199+      "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" ls-files --deleted | while IFS= read -r file; do
200+          "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" checkout -- ":/$file"
201       done
202 
203-      if [ -n "$("$GIT_PROGRAM" ls-files --modified)" ]; then
204+      if [ -n "$("/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" ls-files --modified)" ]; then
205         local msg
206         IFS='' read -r -d '' msg <<EOF
207 **NOTE**
208@@ -846,7 +846,7 @@ EOF
209 
210     # operate on the yadm repo's configuration file
211     # this is always local to the machine
212-    "$GIT_PROGRAM" config "$@"
213+    "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" config "$@"
214 
215     CHANGES_POSSIBLE=1
216 
217@@ -854,7 +854,7 @@ EOF
218     # make sure parent folder of config file exists
219     assert_parent "$YADM_CONFIG"
220     # operate on the yadm configuration file
221-    "$GIT_PROGRAM" config --file="$(mixed_path "$YADM_CONFIG")" "$@"
222+    "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" config --file="$(mixed_path "$YADM_CONFIG")" "$@"
223 
224   fi
225 
226@@ -906,13 +906,13 @@ function _decrypt_from() {
227   case "$yadm_cipher" in
228     gpg)
229       require_gpg
230-      $GPG_PROGRAM -d "$output_archive"
231+      /nix/store/y2hcv73jwa5k1s1dhj65mjz4cwyrfzm8-gnupg-2.2.27/bin/gpg -d "$output_archive"
232       ;;
233 
234     openssl)
235       require_openssl
236       _set_openssl_options
237-      $OPENSSL_PROGRAM enc -d "${OPENSSL_OPTS[@]}" -in "$output_archive"
238+      /nix/store/gdrl09cxsbml1gdb0dqkjb11i1s36183-openssl-1.1.1k-bin/bin/openssl enc -d "${OPENSSL_OPTS[@]}" -in "$output_archive"
239       ;;
240 
241     *)
242@@ -933,13 +933,13 @@ function _encrypt_to() {
243     gpg)
244       require_gpg
245       _set_gpg_options
246-      $GPG_PROGRAM --yes "${GPG_OPTS[@]}" --output "$output_archive"
247+      /nix/store/y2hcv73jwa5k1s1dhj65mjz4cwyrfzm8-gnupg-2.2.27/bin/gpg --yes "${GPG_OPTS[@]}" --output "$output_archive"
248       ;;
249 
250     openssl)
251       require_openssl
252       _set_openssl_options
253-      $OPENSSL_PROGRAM enc -e "${OPENSSL_OPTS[@]}" -out "$output_archive"
254+      /nix/store/gdrl09cxsbml1gdb0dqkjb11i1s36183-openssl-1.1.1k-bin/bin/openssl enc -e "${OPENSSL_OPTS[@]}" -out "$output_archive"
255       ;;
256 
257     *)
258@@ -963,7 +963,7 @@ function decrypt() {
259   fi
260 
261   # decrypt the archive
262-  if (_decrypt_from "$YADM_ARCHIVE" || echo 1) | tar v${tar_option}f - -C "$YADM_WORK"; then
263+  if (_decrypt_from "$YADM_ARCHIVE" || echo 1) | /nix/store/gh440dlrbabj33lj410w37qpjfidhgnd-gnutar-1.34/bin/tar v${tar_option}f - -C "$YADM_WORK"; then
264     [ ! "$DO_LIST" = "YES" ] && echo "All files decrypted."
265   else
266     error_out "Unable to extract encrypted files."
267@@ -987,21 +987,21 @@ function encrypt() {
268   echo
269 
270   # encrypt all files which match the globs
271-  if tar -f - -c "${ENCRYPT_INCLUDE_FILES[@]}" | _encrypt_to "$YADM_ARCHIVE"; then
272+  if /nix/store/gh440dlrbabj33lj410w37qpjfidhgnd-gnutar-1.34/bin/tar -f - -c "${ENCRYPT_INCLUDE_FILES[@]}" | _encrypt_to "$YADM_ARCHIVE"; then
273     echo "Wrote new file: $YADM_ARCHIVE"
274   else
275     error_out "Unable to write $YADM_ARCHIVE"
276   fi
277 
278   # offer to add YADM_ARCHIVE if untracked
279-  archive_status=$("$GIT_PROGRAM" status --porcelain -uall "$(mixed_path "$YADM_ARCHIVE")" 2>/dev/null)
280+  archive_status=$("/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" status --porcelain -uall "$(mixed_path "$YADM_ARCHIVE")" 2>/dev/null)
281   archive_regex="^\?\?"
282   if [[ $archive_status =~ $archive_regex ]] ; then
283     echo "It appears that $YADM_ARCHIVE is not tracked by yadm's repository."
284     echo "Would you like to add it now? (y/n)"
285     read -r answer < /dev/tty
286     if [[ $answer =~ ^[yY]$ ]] ; then
287-      "$GIT_PROGRAM" add "$(mixed_path "$YADM_ARCHIVE")"
288+      "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" add "$(mixed_path "$YADM_ARCHIVE")"
289     fi
290   fi
291 
292@@ -1083,8 +1083,8 @@ function git_command() {
293   CHANGES_POSSIBLE=1
294 
295   # pass commands through to git
296-  debug "Running git command $GIT_PROGRAM $*"
297-  "$GIT_PROGRAM" "$@"
298+  debug "Running git command git $*"
299+  "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" "$@"
300   return "$?"
301 }
302 
303@@ -1139,13 +1139,13 @@ function init() {
304   # remove existing if forcing the init to happen anyway
305   [ -d "$YADM_REPO" ] && {
306     debug "Removing existing repo prior to init"
307-    "$GIT_PROGRAM" -C "$YADM_WORK" submodule deinit -f --all
308-    rm -rf "$YADM_REPO"
309+    "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" -C "$YADM_WORK" submodule deinit -f --all
310+    /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/rm -rf "$YADM_REPO"
311   }
312 
313   # init a new bare repo
314   debug "Init new repo"
315-  "$GIT_PROGRAM" init --shared=0600 --bare "$(mixed_path "$YADM_REPO")" "$@"
316+  "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" init --shared=0600 --bare "$(mixed_path "$YADM_REPO")" "$@"
317   configure_repo
318 
319   CHANGES_POSSIBLE=1
320@@ -1239,7 +1239,7 @@ function list() {
321   fi
322 
323   # list tracked files
324-  "$GIT_PROGRAM" ls-files
325+  "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" ls-files
326 
327 }
328 
329@@ -1276,7 +1276,7 @@ function perms() {
330   # remove group/other permissions from collected globs
331   #shellcheck disable=SC2068
332   #(SC2068 is disabled because in this case, we desire globbing)
333-  chmod -f go-rwx ${GLOBS[@]} &> /dev/null
334+  /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/chmod -f go-rwx ${GLOBS[@]} &> /dev/null
335   # TODO: detect and report changing permissions in a portable way
336 
337 }
338@@ -1309,18 +1309,18 @@ function upgrade() {
339 
340       # Must absorb git dirs, otherwise deinit below will fail for modules that have
341       # been cloned first and then added as a submodule.
342-      "$GIT_PROGRAM" submodule absorbgitdirs
343+      "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" submodule absorbgitdirs
344 
345       local submodule_status
346-      submodule_status=$("$GIT_PROGRAM" -C "$YADM_WORK" submodule status)
347+      submodule_status=$("/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" -C "$YADM_WORK" submodule status)
348       while read -r sha submodule rest; do
349           [ "$submodule" == "" ] && continue
350           if [[ "$sha" = -* ]]; then
351               continue
352           fi
353-          "$GIT_PROGRAM" -C "$YADM_WORK" submodule deinit ${FORCE:+-f} -- "$submodule" || {
354+          "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" -C "$YADM_WORK" submodule deinit ${FORCE:+-f} -- "$submodule" || {
355               for other in "${submodules[@]}"; do
356-                  "$GIT_PROGRAM" -C "$YADM_WORK" submodule update --init --recursive -- "$other"
357+                  "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" -C "$YADM_WORK" submodule update --init --recursive -- "$other"
358               done
359               error_out "Unable to upgrade. Could not deinit submodule $submodule"
360           }
361@@ -1328,7 +1328,7 @@ function upgrade() {
362       done <<< "$submodule_status"
363 
364       assert_parent "$YADM_REPO"
365-      mv "$LEGACY_REPO" "$YADM_REPO"
366+      /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mv "$LEGACY_REPO" "$YADM_REPO"
367     fi
368   fi
369   GIT_DIR="$YADM_REPO"
370@@ -1345,10 +1345,10 @@ function upgrade() {
371     echo "Moving $LEGACY_ARCHIVE to $YADM_ARCHIVE"
372     assert_parent "$YADM_ARCHIVE"
373     # test to see if path is "tracked" in repo, if so 'git mv' must be used
374-    if "$GIT_PROGRAM" ls-files --error-unmatch "$LEGACY_ARCHIVE" &> /dev/null; then
375-      "$GIT_PROGRAM" mv "$LEGACY_ARCHIVE" "$YADM_ARCHIVE" && repo_updates=1
376+    if "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" ls-files --error-unmatch "$LEGACY_ARCHIVE" &> /dev/null; then
377+      "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" mv "$LEGACY_ARCHIVE" "$YADM_ARCHIVE" && repo_updates=1
378     else
379-      mv -i "$LEGACY_ARCHIVE" "$YADM_ARCHIVE"
380+      /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mv -i "$LEGACY_ARCHIVE" "$YADM_ARCHIVE"
381     fi
382   fi
383 
384@@ -1367,17 +1367,17 @@ function upgrade() {
385       echo "Moving $legacy_path to $new_filename"
386       assert_parent "$new_filename"
387       # test to see if path is "tracked" in repo, if so 'git mv' must be used
388-      if "$GIT_PROGRAM" ls-files --error-unmatch "$legacy_path" &> /dev/null; then
389-        "$GIT_PROGRAM" mv "$legacy_path" "$new_filename" && repo_updates=1
390+      if "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" ls-files --error-unmatch "$legacy_path" &> /dev/null; then
391+        "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" mv "$legacy_path" "$new_filename" && repo_updates=1
392       else
393-        mv -i "$legacy_path" "$new_filename"
394+        /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mv -i "$legacy_path" "$new_filename"
395       fi
396     fi
397   done
398 
399   # handle submodules, which need to be reinitialized
400   for submodule in "${submodules[@]}"; do
401-      "$GIT_PROGRAM" -C "$YADM_WORK" submodule update --init --recursive -- "$submodule"
402+      "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" -C "$YADM_WORK" submodule update --init --recursive -- "$submodule"
403   done
404 
405   [ "$actions_performed" -eq 0 ] && \
406@@ -1647,7 +1647,7 @@ function configure_paths() {
407   # obtain YADM_WORK from repo if it exists
408   if [ -d "$GIT_DIR" ]; then
409     local work
410-    work=$(unix_path "$("$GIT_PROGRAM" config core.worktree)")
411+    work=$(unix_path "$("/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" config core.worktree)")
412     [ -n "$work" ] && YADM_WORK="$work"
413   fi
414 
415@@ -1666,16 +1666,16 @@ function configure_repo() {
416   debug "Configuring new repo"
417 
418   # change bare to false (there is a working directory)
419-  "$GIT_PROGRAM" config core.bare 'false'
420+  "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" config core.bare 'false'
421 
422   # set the worktree for the yadm repo
423-  "$GIT_PROGRAM" config core.worktree "$(mixed_path "$YADM_WORK")"
424+  "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" config core.worktree "$(mixed_path "$YADM_WORK")"
425 
426   # by default, do not show untracked files and directories
427-  "$GIT_PROGRAM" config status.showUntrackedFiles no
428+  "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" config status.showUntrackedFiles no
429 
430   # possibly used later to ensure we're working on the yadm repo
431-  "$GIT_PROGRAM" config yadm.managed 'true'
432+  "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" config yadm.managed 'true'
433 
434 }
435 
436@@ -1684,16 +1684,16 @@ function set_operating_system() {
437   if [[ "$(<$PROC_VERSION)" =~ [Mm]icrosoft ]]; then
438     OPERATING_SYSTEM="WSL"
439   else
440-    OPERATING_SYSTEM=$(uname -s)
441+    OPERATING_SYSTEM=$(/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/uname -s)
442   fi 2>/dev/null
443 
444   case "$OPERATING_SYSTEM" in
445     CYGWIN*|MINGW*|MSYS*)
446-      git_version="$("$GIT_PROGRAM" --version 2>/dev/null)"
447+      git_version="$("/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" --version 2>/dev/null)"
448       if [[ "$git_version" =~ windows ]] ; then
449           USE_CYGPATH=1
450       fi
451-      OPERATING_SYSTEM=$(uname -o)
452+      OPERATING_SYSTEM=$(/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/uname -o)
453       ;;
454     *)
455       ;;
456@@ -1703,7 +1703,7 @@ function set_operating_system() {
457 
458 function set_awk() {
459   local pgm
460-  for pgm in "${AWK_PROGRAM[@]}"; do
461+  for pgm in "awk"; do
462     command -v "$pgm" &> /dev/null && AWK_PROGRAM=("$pgm") && return
463   done
464 }
465@@ -1800,7 +1800,7 @@ function assert_private_dirs() {
466     if [ ! -d "$YADM_WORK/$private_dir" ]; then
467       debug "Creating $YADM_WORK/$private_dir"
468       #shellcheck disable=SC2174
469-      mkdir -m 0700 -p "$YADM_WORK/$private_dir" &> /dev/null
470+      /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mkdir -m 0700 -p "$YADM_WORK/$private_dir" &> /dev/null
471     fi
472   done
473 }
474@@ -1808,7 +1808,7 @@ function assert_private_dirs() {
475 function assert_parent() {
476   basedir=${1%/*}
477   if [ -n "$basedir" ]; then
478-    [ -e "$basedir" ] || mkdir -p "$basedir"
479+    [ -e "$basedir" ] || /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mkdir -p "$basedir"
480   fi
481 }
482 
483@@ -1816,7 +1816,7 @@ function display_private_perms() {
484   when="$1"
485   for private_dir in $(private_dirs all); do
486     if [ -d "$YADM_WORK/$private_dir" ]; then
487-      private_perms=$(ls -ld "$YADM_WORK/$private_dir")
488+      private_perms=$(/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/ls -ld "$YADM_WORK/$private_dir")
489       debug "$when" private dir perms "$private_perms"
490     fi
491   done
492@@ -1890,7 +1890,7 @@ function parse_encrypt() {
493 
494   # sort the encrypted files
495   #shellcheck disable=SC2207
496-  IFS=$'\n' ENCRYPT_INCLUDE_FILES=($(LC_ALL=C sort <<<"${FINAL_INCLUDE[*]}"))
497+  IFS=$'\n' ENCRYPT_INCLUDE_FILES=($(LC_ALL=C /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/sort <<<"${FINAL_INCLUDE[*]}"))
498   unset IFS
499 
500   if [ "$unset_globstar" = "1" ]; then
501@@ -2016,10 +2016,10 @@ function get_mode {
502   local mode
503 
504   # most *nixes
505-  mode=$(stat -c '%a' "$filename" 2>/dev/null)
506+  mode=$(/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/stat -c '%a' "$filename" 2>/dev/null)
507   if [ -z "$mode" ] ; then
508     # BSD-style
509-    mode=$(stat -f '%p' "$filename" 2>/dev/null)
510+    mode=$(/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/stat -f '%p' "$filename" 2>/dev/null)
511     mode=${mode: -4}
512   fi
513 
514@@ -2035,7 +2035,7 @@ function copy_perms {
515   local source="$1"
516   local dest="$2"
517   mode=$(get_mode "$source")
518-  [ -n "$mode" ] && chmod "$mode" "$dest"
519+  [ -n "$mode" ] && /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/chmod "$mode" "$dest"
520   return 0
521 }
522 
523@@ -2063,8 +2063,8 @@ function require_git() {
524     GIT_PROGRAM="$alt_git"
525     more_info="\nThis command has been set via the yadm.git-program configuration."
526   fi
527-  command -v "$GIT_PROGRAM" &> /dev/null ||
528-    error_out "This functionality requires Git to be installed, but the command '$GIT_PROGRAM' cannot be located.$more_info"
529+  command -v "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" &> /dev/null ||
530+    error_out "This functionality requires Git to be installed, but the command 'git' cannot be located.$more_info"
531 }
532 function require_gpg() {
533   local alt_gpg
534@@ -2076,8 +2076,8 @@ function require_gpg() {
535     GPG_PROGRAM="$alt_gpg"
536     more_info="\nThis command has been set via the yadm.gpg-program configuration."
537   fi
538-  command -v "$GPG_PROGRAM" &> /dev/null ||
539-    error_out "This functionality requires GPG to be installed, but the command '$GPG_PROGRAM' cannot be located.$more_info"
540+  command -v "/nix/store/y2hcv73jwa5k1s1dhj65mjz4cwyrfzm8-gnupg-2.2.27/bin/gpg" &> /dev/null ||
541+    error_out "This functionality requires GPG to be installed, but the command 'gpg' cannot be located.$more_info"
542 }
543 function require_openssl() {
544   local alt_openssl
545@@ -2089,8 +2089,8 @@ function require_openssl() {
546     OPENSSL_PROGRAM="$alt_openssl"
547     more_info="\nThis command has been set via the yadm.openssl-program configuration."
548   fi
549-  command -v "$OPENSSL_PROGRAM" &> /dev/null ||
550-    error_out "This functionality requires OpenSSL to be installed, but the command '$OPENSSL_PROGRAM' cannot be located.$more_info"
551+  command -v "/nix/store/gdrl09cxsbml1gdb0dqkjb11i1s36183-openssl-1.1.1k-bin/bin/openssl" &> /dev/null ||
552+    error_out "This functionality requires OpenSSL to be installed, but the command 'openssl' cannot be located.$more_info"
553 }
554 function require_repo() {
555   [ -d "$YADM_REPO" ] || error_out "Git repo does not exist. did you forget to run 'init' or 'clone'?"
556@@ -2111,11 +2111,11 @@ function bootstrap_available() {
557   return 1
558 }
559 function awk_available() {
560-  command -v "${AWK_PROGRAM[0]}" &> /dev/null && return
561+  command -v "/nix/store/8kpxw5na07ggdl2bs8kiwysif7120r6g-gawk-5.1.0/bin/awk" &> /dev/null && return
562   return 1
563 }
564 function j2cli_available() {
565-  command -v "$J2CLI_PROGRAM" &> /dev/null && return
566+  command -v "/nix/store/p0876isrv4gi1pbfkc6bms737grzwjlj-python3.8-j2cli-0.3.10/bin/j2" &> /dev/null && return
567   return 1
568 }
569 function envtpl_available() {
570@@ -2123,11 +2123,11 @@ function envtpl_available() {
571   return 1
572 }
573 function esh_available() {
574-  command -v "$ESH_PROGRAM" &> /dev/null && return
575+  command -v "/nix/store/irnmflhx17rlhzvz6g6d3f788aykf571-esh-0.1.1/bin/esh" &> /dev/null && return
576   return 1
577 }
578 function readlink_available() {
579-  command -v "readlink" &> /dev/null && return
580+  command -v "/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/readlink" &> /dev/null && return
581   return 1
582 }
583 
584@@ -2175,3 +2175,41 @@ if [ "$YADM_TEST" != 1 ] ; then
585   configure_paths
586   main "${MAIN_ARGS[@]}"
587 fi
588+
589+### resholve directives (auto-generated) ## format_version: 2
590+# resholve: fake external:cygpath
591+# resholve: fix $AWK_PROGRAM:awk
592+# resholve: fix $ESH_PROGRAM:esh
593+# resholve: fix $GIT_PROGRAM:git
594+# resholve: fix $GPG_PROGRAM:gpg
595+# resholve: fix $J2CLI_PROGRAM:j2
596+# resholve: fix $OPENSSL_PROGRAM:openssl
597+# resholve: keep $ENVTPL_PROGRAM
598+# resholve: keep $LSB_RELEASE_PROGRAM
599+# resholve: keep $SHELL
600+# resholve: keep $YADM_COMMAND
601+# resholve: keep $hook_command
602+# resholve: keep $template_cmd
603+# resholve: keep /nix/store/8kpxw5na07ggdl2bs8kiwysif7120r6g-gawk-5.1.0/bin/awk
604+# resholve: keep /nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git
605+# resholve: keep /nix/store/gdrl09cxsbml1gdb0dqkjb11i1s36183-openssl-1.1.1k-bin/bin/openssl
606+# resholve: keep /nix/store/gh440dlrbabj33lj410w37qpjfidhgnd-gnutar-1.34/bin/tar
607+# resholve: keep /nix/store/irnmflhx17rlhzvz6g6d3f788aykf571-esh-0.1.1/bin/esh
608+# resholve: keep /nix/store/p0876isrv4gi1pbfkc6bms737grzwjlj-python3.8-j2cli-0.3.10/bin/j2
609+# resholve: keep /nix/store/pcjan45rssdn01cxx3sjg70avjg6c3ni-bash-4.4-p23/bin/bash
610+# resholve: keep /nix/store/y2hcv73jwa5k1s1dhj65mjz4cwyrfzm8-gnupg-2.2.27/bin/gpg
611+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/chmod
612+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/cp
613+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/dirname
614+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/id
615+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/ln
616+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/ls
617+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mkdir
618+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mv
619+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/readlink
620+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/rm
621+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/sort
622+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/stat
623+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/uname
624+# resholve: keep exec:$YADM_BOOTSTRAP
625+
Discussing this elsewhere?
Enter 'link' for a markdown link
or 'tweet <message>' for a pre-populated Tweet :)
Want to subscribe? Enter 'rss' for a feed URL.
>