I've been working on resholve to improve Nix packaging for shell projects since early 2020, but the new v0.6.0 release is a decent leap forward in the fraction of Shell scripts and programs resholve can handle. There's a more basic introduction to using resholve in my previous post (write a simple shell package with resholve), but some readers were interested in seeing a bit more detail on using resholve. As I'm pulling the release together and waiting for some feedback on release candidates, I took a little time to record a screencast where I walk through using resholve to re-package a complex Shell project.
The video is about 36 minutes long, though, so I also wanted to summarize it for anyone who is curious, but not 36-minutes curious. :)
If you want to follow along with the post, you can clone the source with:
1git clone https://gist.github.com/75978730de1715570e145a8073b4d651.git
Note: Nix sources in the video and post slightly differ. In the video I work directly on the nixpkgs source, but the post is using standalone Nix expressions.
resholve
is a command-line tool
that helps you find
and resolve
external dependencies
in Shell scripts.
The main reason
you'd want to do this
is to make Shell
easier to package or deploy,
but the need for another tool
may not be evident
if you don't spend much time
dealing with Shell.
The flexibility
that makes Shell a good glue language
also makes it tricky to package.
Without a build process to gripe at you
it's even hard to know
whether you have identified
all of the external dependencies
of a complex Shell program
--especially one
that SOURCES a lot of other files
and uses a lot of functions.
Even if
a dependency is installed
the user's PATH
may not include the right directories,
or they may be in the wrong order,
or the script's author
may have hard-coded
absolute paths
that aren't right for your OS
or package manager.
Before I jump in to converting the package
I want to show, succinctly, what's wrong
with YADM's existing package by trying to run yadm in a pure nix shell (which only includes yadm, and Nix's set of standard build tools) on the PATH.
missing_git.console
1bash-5.1 $ nix-shell --pure -p yadm --run yadm
2/nix/store/7agjh6y20b2pn17sp9hrlx07f33y23wm-yadm-3.1.0/bin/yadm: line 1650: git: command not found
3/nix/store/7agjh6y20b2pn17sp9hrlx07f33y23wm-yadm-3.1.0/bin/yadm: line 857: git: command not found
4ERROR: This functionality requires Git to be installed, but the command 'git' cannot be located.
YADM is a git-based dotfile manager, so the invocation breaks because it expects to find git via the PATH and I didn't include the git package.
Here's the source for the existing package:
yadm_before.nix
1{ pkgs ? import <nixpkgs> { } }:
2
3with pkgs;
4stdenv.mkDerivation rec {
5 pname = "yadm";
6 version = "3.1.0";
7
8 buildInputs = [ git gnupg ];
9
10 nativeBuildInputs = [ installShellFiles ];
11
12 src = fetchFromGitHub {
13 owner = "TheLocehiliosan";
14 repo = "yadm";
15 rev = version;
16 sha256 = "0ga0p28nvqilswa07bzi93adk7wx6d5pgxlacr9wl9v1h6cds92s";
17 };
18
19 dontConfigure = true;
20 dontBuild = true;
21
22 installPhase = ''
23 runHook preInstall
24 install -Dt $out/bin yadm
25 runHook postInstall
26 '';
27
28 postInstall = ''
29 installManPage yadm.1
30 installShellCompletion --cmd yadm \
31 --zsh completion/zsh/_yadm \
32 --bash completion/bash/yadm
33 '';
34
35 meta = {
36 homepage = "https://github.com/TheLocehiliosan/yadm";
37 description = "Yet Another Dotfiles Manager";
38 longDescription = ''
39 yadm is a dotfile management tool with 3 main features:
40 * Manages files across systems using a single Git repository.
41 * Provides a way to use alternate files on a specific OS or host.
42 * Supplies a method of encrypting confidential data so it can safely be stored in your repository.
43 '';
44 license = lib.licenses.gpl3Plus;
45 maintainers = with lib.maintainers; [ abathur ];
46 platforms = lib.platforms.unix;
47 };
48}
49
The highlighted line
shows that git
is in the package's buildInputs,
but buildInputs
don't get pulled into the PATH
with the package.
Only the package itself
is added to the PATH.
This is fine
for users whose PATH
already contains
a compatible version
of git.
But,
as with Shell scripts and programs everywhere
yadm's Nix package
would be happy to misfire
if you (or a script, or a daemon)
used it with the wrong environment.
The Nix ecosystem
already has
a few ways
to address
this kind of problem,
including
patching
Shell wrappers
and the propagatedUserEnvPkgs
attribute.
Each has its own problems
but they share
a fatal flaw:
all of them
require you to know
the script's dependencies.
resholve's goal
is to spot
the most-common issues here,
give you a toolkit for triaging them,
and exit with an error until you do.
In this section I'll zoom in on a few
of the most-important conversion steps,
but I won't discuss every last step as
the video does.
Here's the source after the initial conversion to use the resholve API. This is, most-succinctly, the part where I actually convert it to use resholve:
yadm_resholving_1.nix
1{ pkgs ? import <nixpkgs> { } }:
2
3with pkgs;
4resholvePackage rec {
5 pname = "yadm";
6 version = "3.1.0";
7
8 nativeBuildInputs = [ installShellFiles ];
9
10 src = fetchFromGitHub {
11 owner = "TheLocehiliosan";
12 repo = "yadm";
13 rev = version;
14 sha256 = "0ga0p28nvqilswa07bzi93adk7wx6d5pgxlacr9wl9v1h6cds92s";
15 };
16
17 dontConfigure = true;
18 dontBuild = true;
19
20 installPhase = ''
21 runHook preInstall
22 install -Dt $out/bin yadm
23 runHook postInstall
24 '';
25
26 postInstall = ''
27 installManPage yadm.1
28 installShellCompletion --cmd yadm \
29 --zsh completion/zsh/_yadm \
30 --bash completion/bash/yadm
31 '';
32
33 solutions = {
34 yadm = {
35 scripts = [ "bin/yadm" ];
36 interpreter = "${bash}/bin/sh";
37 inputs = [
38 git
39 ];
40 };
41 };
42
43 passthru.tests = {
44 minimal = runCommand "${pname}-test" {} ''
45 export HOME=$out
46 ${yadm}/bin/yadm init
47 '';
48 };
49
50 meta = {
51 homepage = "https://github.com/TheLocehiliosan/yadm";
52 description = "Yet Another Dotfiles Manager";
53 longDescription = ''
54 yadm is a dotfile management tool with 3 main features:
55 * Manages files across systems using a single Git repository.
56 * Provides a way to use alternate files on a specific OS or host.
57 * Supplies a method of encrypting confidential data so it can safely be stored in your repository.
58 '';
59 license = lib.licenses.gpl3Plus;
60 maintainers = with lib.maintainers; [ abathur ];
61 platforms = lib.platforms.unix;
62 };
63}
The rest of the conversion process is basically working in a build-understand-triage loop with resholve to tell it what to do about the things it identifies. I started off from ~scratch, including only the git
dependency that I've already identified.
Here's what resholve kicks out:
build_resholving_1.console
1bash-5.1 $ nix-build yadm_resholving_1.nix --out-link result_resholving_1
2this derivation will be built:
3 /nix/store/8gycx0gqm88b2g645z8v0cwl9ba4bwss-yadm-3.1.0.drv
4building '/nix/store/8gycx0gqm88b2g645z8v0cwl9ba4bwss-yadm-3.1.0.drv'...
5unpacking sources
6unpacking source archive /nix/store/swy5n5gagcwygj7w1gjlaf5g1rbklll7-source
7source root is source
8patching sources
9installing
10post-installation fixup
11[resholve context] : changing directory to /nix/store/12xg9d5y4qhb53hagnk7fr3fcv2hv6kg-yadm-3.1.0
12[resholve context] RESHOLVE_LORE=/nix/store/rs98ww426yzzkmln4bfqrimn792b55nn-more-binlore
13[resholve context] RESHOLVE_INPUTS=/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin
14[resholve context] RESHOLVE_INTERPRETER=/nix/store/pcjan45rssdn01cxx3sjg70avjg6c3ni-bash-4.4-p23/bin/sh
15[resholve context] resholve --overwrite bin/yadm
16 $YADM_COMMAND "${YADM_ARGS[@]}"
17 ^~~~~~~~~~~~~
18/nix/store/12xg9d5y4qhb53hagnk7fr3fcv2hv6kg-yadm-3.1.0/bin/yadm:147: Can't resolve dynamic command
19error: builder for '/nix/store/8gycx0gqm88b2g645z8v0cwl9ba4bwss-yadm-3.1.0.drv' failed with exit code 7
I updated resholvePackage
to let us know
how it's invoking resholve
in case we need to manually debug anything.
resholve is complaining
because the script invokes
whatever is in the variable
YADM_COMMAND.
Since the command may be dynamic
resholve makes us triage.
To figure out what to do with it, I go take a look at how yadm uses this variable:
lines 80-163 of yadm
80function main() {
81
82 require_git
83
84 # capture full command, for passing to hooks
85 # the parameters will be space delimited and
86 # spaces, tabs, and backslashes will be escaped
87 _tab=$'\t'
88 for param in "$@"; do
89 param="${param//\\/\\\\}"
90 param="${param//$_tab/\\$_tab}"
91 param="${param// /\\ }"
92 _fc+=( "$param" )
93 done
94 FULL_COMMAND="${_fc[*]}"
95
96 # create the YADM_DIR & YADM_DATA if they doesn't exist yet
97 [ -d "$YADM_DIR" ] || mkdir -p "$YADM_DIR"
98 [ -d "$YADM_DATA" ] || mkdir -p "$YADM_DATA"
99
100 # parse command line arguments
101 local retval=0
102 internal_commands="^(alt|bootstrap|clean|clone|config|decrypt|encrypt|enter|git-crypt|help|--help|init|introspect|list|perms|transcrypt|upgrade|version|--version)$"
103 if [ -z "$*" ] ; then
104 # no argumnts will result in help()
105 help
106 elif [[ "$1" =~ $internal_commands ]] ; then
107 # for internal commands, process all of the arguments
108 YADM_COMMAND="${1//-/_}"
109 YADM_COMMAND="${YADM_COMMAND/__/}"
110 YADM_ARGS=()
111 shift
112
113 # commands listed below do not process any of the parameters
114 if [[ "$YADM_COMMAND" =~ ^(enter|git_crypt)$ ]] ; then
115 YADM_ARGS=("$@")
116 else
117 while [[ $# -gt 0 ]] ; do
118 key="$1"
119 case $key in
120 -a) # used by list()
121 LIST_ALL="YES"
122 ;;
123 -d) # used by all commands
124 DEBUG="YES"
125 ;;
126 -f) # used by init(), clone() and upgrade()
127 FORCE="YES"
128 ;;
129 -l) # used by decrypt()
130 DO_LIST="YES"
131 [[ "$YADM_COMMAND" =~ ^(clone|config)$ ]] && YADM_ARGS+=("$1")
132 ;;
133 -w) # used by init() and clone()
134 YADM_WORK="$(qualify_path "$2" "work tree")"
135 shift
136 ;;
137 *) # any unhandled arguments
138 YADM_ARGS+=("$1")
139 ;;
140 esac
141 shift
142 done
143 fi
144 [ ! -d "$YADM_WORK" ] && error_out "Work tree does not exist: [$YADM_WORK]"
145 HOOK_COMMAND="$YADM_COMMAND"
146 invoke_hook "pre"
147 $YADM_COMMAND "${YADM_ARGS[@]}"
148 else
149 # any other commands are simply passed through to git
150 HOOK_COMMAND="$1"
151 invoke_hook "pre"
152 git_command "$@"
153 retval="$?"
154 fi
155
156 # process automatic events
157 auto_alt
158 auto_perms
159 auto_bootstrap
160
161 exit_with_hook $retval
162
163}
The comment above the variable definition
tells us this is for internal commands
and the comment is backed up
by the fact that it's set
from the value of parameter 1
in the main function.
Since the value comes from the user
at runtime
the best way to triage this invocation
is it keep it by adding a keep directive to the solution:
1keep = {
2 "$YADM_COMMAND" = true;
3};
build_resholving_2.console
1bash-5.1 $ nix-build yadm_resholving_2.nix --out-link result_resholving_2
2this derivation will be built:
3 /nix/store/7j28v1zf2k2677b262bsa1z1am2hs23h-yadm-3.1.0.drv
4building '/nix/store/7j28v1zf2k2677b262bsa1z1am2hs23h-yadm-3.1.0.drv'...
5unpacking sources
6unpacking source archive /nix/store/swy5n5gagcwygj7w1gjlaf5g1rbklll7-source
7source root is source
8patching sources
9installing
10post-installation fixup
11[resholve context] : changing directory to /nix/store/5c04gzx1hhwx7pmqjmf7s7kf72a8l473-yadm-3.1.0
12[resholve context] RESHOLVE_LORE=/nix/store/rs98ww426yzzkmln4bfqrimn792b55nn-more-binlore
13[resholve context] RESHOLVE_INPUTS=/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin
14[resholve context] RESHOLVE_INTERPRETER=/nix/store/pcjan45rssdn01cxx3sjg70avjg6c3ni-bash-4.4-p23/bin/sh
15[resholve context] RESHOLVE_KEEP='$YADM_COMMAND'
16[resholve context] resholve --overwrite bin/yadm
17 "${AWK_PROGRAM[0]}" \
18 ^
19/nix/store/5c04gzx1hhwx7pmqjmf7s7kf72a8l473-yadm-3.1.0/bin/yadm:425: Can't resolve dynamic command
20error: builder for '/nix/store/7j28v1zf2k2677b262bsa1z1am2hs23h-yadm-3.1.0.drv' failed with exit code 7
After rebuilding, resholve spits out the same basic error--though something named AWK_PROGRAM
probably contains an external command.
Once again, I jump into the yadm source to see how it's being used. Here's its declaration, alongside some similar ones:
lines 47-56 of yadm
47GPG_PROGRAM="gpg"
48OPENSSL_PROGRAM="openssl"
49GIT_PROGRAM="git"
50AWK_PROGRAM=("gawk" "awk")
51GIT_CRYPT_PROGRAM="git-crypt"
52TRANSCRYPT_PROGRAM="transcrypt"
53J2CLI_PROGRAM="j2"
54ENVTPL_PROGRAM="envtpl"
55ESH_PROGRAM="esh"
56LSB_RELEASE_PROGRAM="lsb_release"
Here's the actual invocation using it:
lines 425-434 of yadm
425 "${AWK_PROGRAM[0]}" \
426 -v class="$local_class" \
427 -v os="$local_system" \
428 -v host="$local_host" \
429 -v user="$local_user" \
430 -v distro="$local_distro" \
431 -v source="$input" \
432 -v source_dir="$(dirname "$input")" \
433 "$awk_pgm" \
434 "$input" > "$temp_file" || rm -f "$temp_file"
And here's a ~setup function that also depends on the variable:
lines 1704-1709 of yadm
1704function set_awk() {
1705 local pgm
1706 for pgm in "${AWK_PROGRAM[@]}"; do
1707 command -v "$pgm" &> /dev/null && AWK_PROGRAM=("$pgm") && return
1708 done
1709}
The ability to fix this practice of storing commands in variables is one of the new features in this release. Narrowly, I can fix resholve's complaint by adding gawk
to this solution's inputs, along with a fix directive like:
1fix = {
2 "$AWK_PROGRAM" = [ "awk" ];
3}
This fix directive tells resholve
to replace
parameter expansions
of AWK_PROGRAM
with awk
. It will
replace the entire expansion
(even if it's a complex braced one)
before it resolves commands. Instances
that are commands will get resolved
later, and others will keep the bare
replacement. This tries to
strike a good balance between getting
the commands resolved without having
to worry about what else the script
does with the variable.
As you saw in the yadm source, awk is only
one of several commands that get this
treatment. For the next step, I'll go ahead and add all of these, plus all of the remaining ~dynamic command instances like YADM_COMMAND.
yadm_resholving_2-3.diff
1bash-5.1 $ git diff --no-index yadm_resholving_2.nix yadm_resholving_3.nix
2diff --git a/yadm_resholving_2.nix b/yadm_resholving_3.nix
3index 90fc523..5d8f5b2 100644
4--- a/yadm_resholving_2.nix
5+++ b/yadm_resholving_3.nix
6@@ -36,9 +36,47 @@ resholvePackage rec {
7 interpreter = "${bash}/bin/sh";
8 inputs = [
9 git
10+ gnupg
11+ openssl
12+ gawk
13+ /*
14+ TODO: yadm can use git-crypt and transcrypt
15+ but it does so in a way that resholve 0.6.0
16+ can't yet do anything smart about. It looks
17+ like these are for interactive use, so the
18+ main impact should just be that users still
19+ need both of these packages in their profile
20+ to support their use in yadm.
21+ */
22+ # git-crypt
23+ # transcrypt
24+ j2cli
25+ esh
26 ];
27+ fix = {
28+ "$GPG_PROGRAM" = [ "gpg" ];
29+ "$OPENSSL_PROGRAM" = [ "openssl" ];
30+ "$GIT_PROGRAM" = [ "git" ];
31+ "$AWK_PROGRAM" = [ "awk" ];
32+ # see inputs comment
33+ # "$GIT_CRYPT_PROGRAM" = [ "git-crypt" ];
34+ # "$TRANSCRYPT_PROGRAM" = [ "transcrypt" ];
35+ "$J2CLI_PROGRAM" = [ "j2" ];
36+ "$ESH_PROGRAM" = [ "esh" ];
37+ # not in nixpkgs
38+ # "$ENVTPL_PROGRAM" = [ "envtpl" ];
39+ # "$LSB_RELEASE_PROGRAM" = [ "lsb_release" ];
40+ };
41 keep = {
42- "$YADM_COMMAND" = true;
43+ "$YADM_COMMAND" = true; # internal cmds
44+ "$template_cmd" = true; # dynamic, template-engine
45+ "$SHELL" = true; # probably user env? unsure
46+ "$hook_command" = true; # ~git hooks?
47+ "exec" = [ "$YADM_BOOTSTRAP" ]; # yadm bootstrap script
48+
49+ # not in nixpkgs
50+ "$ENVTPL_PROGRAM" = true;
51+ "$LSB_RELEASE_PROGRAM" = true;
52 };
53 };
54 };
build_resholving_3.console
1bash-5.1 $ nix-build yadm_resholving_3.nix --out-link result_resholving_3
2this derivation will be built:
3 /nix/store/4vrkfc5rqdgdnvlcynpljgw97im706yl-yadm-3.1.0.drv
4building '/nix/store/4vrkfc5rqdgdnvlcynpljgw97im706yl-yadm-3.1.0.drv'...
5unpacking sources
6unpacking source archive /nix/store/swy5n5gagcwygj7w1gjlaf5g1rbklll7-source
7source root is source
8patching sources
9installing
10post-installation fixup
11[resholve context] : changing directory to /nix/store/skghr1m65mh2bk47q7dv6f0f131fqfrq-yadm-3.1.0
12[resholve context] RESHOLVE_LORE=/nix/store/vimyjdi52n71xp4h8zmig5zks5dvz5y2-more-binlore
13[resholve context] RESHOLVE_FIX='$AWK_PROGRAM:awk $ESH_PROGRAM:esh $GIT_PROGRAM:git $GPG_PROGRAM:gpg $J2CLI_PROGRAM:j2 $OPENSSL_PROGRAM:openssl'
14[resholve context] RESHOLVE_INPUTS=/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin:/nix/store/y2hcv73jwa5k1s1dhj65mjz4cwyrfzm8-gnupg-2.2.27/bin:/nix/store/gdrl09cxsbml1gdb0dqkjb11i1s36183-openssl-1.1.1k-bin/bin:/nix/store/8kpxw5na07ggdl2bs8kiwysif7120r6g-gawk-5.1.0/bin:/nix/store/p0876isrv4gi1pbfkc6bms737grzwjlj-python3.8-j2cli-0.3.10/bin:/nix/store/irnmflhx17rlhzvz6g6d3f788aykf571-esh-0.1.1/bin
15[resholve context] RESHOLVE_INTERPRETER=/nix/store/pcjan45rssdn01cxx3sjg70avjg6c3ni-bash-4.4-p23/bin/sh
16[resholve context] RESHOLVE_KEEP='$ENVTPL_PROGRAM $LSB_RELEASE_PROGRAM $SHELL $YADM_COMMAND $hook_command $template_cmd exec:$YADM_BOOTSTRAP'
17[resholve context] resholve --overwrite bin/yadm
18 "esh" -o "$temp_file" "$input" \
19 ^
20/nix/store/skghr1m65mh2bk47q7dv6f0f131fqfrq-yadm-3.1.0/bin/yadm:476: 'esh' _might_ be able to execute its arguments, and I don't have any command-specific rules for figuring out if this specific invocation does or not.
21error: builder for '/nix/store/4vrkfc5rqdgdnvlcynpljgw97im706yl-yadm-3.1.0.drv' failed with exit code 9
This error
is part of another new feature.
The update will add support
for recursive resolution.
This was easy enough
for builtins,
but an extra-tricky part
of packaging shell
is that any given external command
might be able
to execute other external commands.
sudo
and xargs
are the most-familiar
examples of this--
but they are far
from the only ones.
To address these, resholve now requires some metadata
for each external command it finds.
It outsources
the job of generating this metadata,
so it's passed in a format I call "lore".
The most important kind of lore
tells resholve
whether an executable
can, cannot, or might
be able to execute its arguments.
resholve's Nix API
automatically supplies lore
for every package in the inputs
using binlore,
the reference lore provider
I'm developing.
This judgement is low-resolution. Binlore
isn't doing advanced binary analysis.
It's using some YARA rules
to sniff for signs
an executable uses an exec API.
When resholve finds invocations
of executables that the lore
indicates "can" or "might"
execute their arguments,
resholve will try to fall back
on internal
command-specific
parsing rules
for finding execed arguments.
If it lacks parsing rules
for the command,
it throws the error
we saw above:
'esh' might be able to execute its arguments,
and I don't have any command-specific rules
for figuring out if this specific invocation does or not.
It'll take some time
(and I'll need some help)
to refine these features
in both binlore and resholve.
But they're a starting point.
For this demo
I just triaged invocations
of each potential execer manually.
If they looked safe, I added a lore override
to tell resholve to assume the executable
is safe. I can override the execer lore for
a single executable like this:
1execer = [
2 "cannot:${j2cli}/bin/j2"
3];
Caution: I'm cutting a corner! If a future
version of the script started using exec
behavior of an executable we override the
lore for, resholve won't be able to help us.
The "best" way to address this is to look into whether
binlore needs an override for this executable, or resholve
needs an argument parser for it.
Once again, I added all of these at once.
For completeness, I'll also demonstrate resholve's most-straightforward use case: resolving simple bare missing commands that can be supplied by an existing package.
build_resholving_4.console
1bash-5.1 $ nix-build yadm_resholving_4.nix --out-link result_resholving_4
2this derivation will be built:
3 /nix/store/idm36d4c16l78f74v6adah3gmqz6zfyq-yadm-3.1.0.drv
4building '/nix/store/idm36d4c16l78f74v6adah3gmqz6zfyq-yadm-3.1.0.drv'...
5unpacking sources
6unpacking source archive /nix/store/swy5n5gagcwygj7w1gjlaf5g1rbklll7-source
7source root is source
8patching sources
9installing
10post-installation fixup
11[resholve context] : changing directory to /nix/store/hirchsqpqm3s8w0jcbgh83gs46qwanwk-yadm-3.1.0
12[resholve context] RESHOLVE_LORE=/nix/store/mxdwp8dxbh7wdn70c051wjl9yzx0nmnc-more-binlore
13[resholve context] RESHOLVE_EXECER='cannot:/nix/store/p0876isrv4gi1pbfkc6bms737grzwjlj-python3.8-j2cli-0.3.10/bin/j2 cannot:/nix/store/irnmflhx17rlhzvz6g6d3f788aykf571-esh-0.1.1/bin/esh cannot:/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git cannot:/nix/store/y2hcv73jwa5k1s1dhj65mjz4cwyrfzm8-gnupg-2.2.27/bin/gpg'
14[resholve context] RESHOLVE_FIX='$AWK_PROGRAM:awk $ESH_PROGRAM:esh $GIT_PROGRAM:git $GPG_PROGRAM:gpg $J2CLI_PROGRAM:j2 $OPENSSL_PROGRAM:openssl'
15[resholve context] RESHOLVE_INPUTS=/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin:/nix/store/y2hcv73jwa5k1s1dhj65mjz4cwyrfzm8-gnupg-2.2.27/bin:/nix/store/gdrl09cxsbml1gdb0dqkjb11i1s36183-openssl-1.1.1k-bin/bin:/nix/store/8kpxw5na07ggdl2bs8kiwysif7120r6g-gawk-5.1.0/bin:/nix/store/p0876isrv4gi1pbfkc6bms737grzwjlj-python3.8-j2cli-0.3.10/bin:/nix/store/irnmflhx17rlhzvz6g6d3f788aykf571-esh-0.1.1/bin:/nix/store/pcjan45rssdn01cxx3sjg70avjg6c3ni-bash-4.4-p23/bin:/nix/store/gh440dlrbabj33lj410w37qpjfidhgnd-gnutar-1.34/bin
16[resholve context] RESHOLVE_INTERPRETER=/nix/store/pcjan45rssdn01cxx3sjg70avjg6c3ni-bash-4.4-p23/bin/sh
17[resholve context] RESHOLVE_KEEP='$ENVTPL_PROGRAM $LSB_RELEASE_PROGRAM $SHELL $YADM_COMMAND $hook_command $template_cmd exec:$YADM_BOOTSTRAP'
18[resholve context] resholve --overwrite bin/yadm
19 [ -d "$YADM_DIR" ] || mkdir -p "$YADM_DIR"
20 ^~~~~
21/nix/store/hirchsqpqm3s8w0jcbgh83gs46qwanwk-yadm-3.1.0/bin/yadm:97: Couldn't resolve command 'mkdir'
22error: builder for '/nix/store/idm36d4c16l78f74v6adah3gmqz6zfyq-yadm-3.1.0.drv' failed with exit code 3
This output indicates that resholve found a bare invocation of mkdir
, but none of the current inputs supply it. In this case, we can solve the error by adding coreutils
to the inputs.
build_resholving_5.console
1bash-5.1 $ nix-build yadm_resholving_5.nix --out-link result_resholving_5
2this derivation will be built:
3 /nix/store/n7m489dav4fi4pwpvsfk0xzk8745782y-yadm-3.1.0.drv
4building '/nix/store/n7m489dav4fi4pwpvsfk0xzk8745782y-yadm-3.1.0.drv'...
5unpacking sources
6unpacking source archive /nix/store/swy5n5gagcwygj7w1gjlaf5g1rbklll7-source
7source root is source
8patching sources
9installing
10post-installation fixup
11[resholve context] : changing directory to /nix/store/xhps4gff69lwdrmm0br3wcdmcw76lz1b-yadm-3.1.0
12[resholve context] RESHOLVE_LORE=/nix/store/1wsp3vhd1p1xjyf3fld01n0xs802v6si-more-binlore
13[resholve context] RESHOLVE_EXECER='cannot:/nix/store/p0876isrv4gi1pbfkc6bms737grzwjlj-python3.8-j2cli-0.3.10/bin/j2 cannot:/nix/store/irnmflhx17rlhzvz6g6d3f788aykf571-esh-0.1.1/bin/esh cannot:/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git cannot:/nix/store/y2hcv73jwa5k1s1dhj65mjz4cwyrfzm8-gnupg-2.2.27/bin/gpg'
14[resholve context] RESHOLVE_FIX='$AWK_PROGRAM:awk $ESH_PROGRAM:esh $GIT_PROGRAM:git $GPG_PROGRAM:gpg $J2CLI_PROGRAM:j2 $OPENSSL_PROGRAM:openssl'
15[resholve context] RESHOLVE_INPUTS=/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin:/nix/store/y2hcv73jwa5k1s1dhj65mjz4cwyrfzm8-gnupg-2.2.27/bin:/nix/store/gdrl09cxsbml1gdb0dqkjb11i1s36183-openssl-1.1.1k-bin/bin:/nix/store/8kpxw5na07ggdl2bs8kiwysif7120r6g-gawk-5.1.0/bin:/nix/store/p0876isrv4gi1pbfkc6bms737grzwjlj-python3.8-j2cli-0.3.10/bin:/nix/store/irnmflhx17rlhzvz6g6d3f788aykf571-esh-0.1.1/bin:/nix/store/pcjan45rssdn01cxx3sjg70avjg6c3ni-bash-4.4-p23/bin:/nix/store/gh440dlrbabj33lj410w37qpjfidhgnd-gnutar-1.34/bin:/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin
16[resholve context] RESHOLVE_INTERPRETER=/nix/store/pcjan45rssdn01cxx3sjg70avjg6c3ni-bash-4.4-p23/bin/sh
17[resholve context] RESHOLVE_KEEP='$ENVTPL_PROGRAM $LSB_RELEASE_PROGRAM $SHELL $YADM_COMMAND $hook_command $template_cmd exec:$YADM_BOOTSTRAP'
18[resholve context] resholve --overwrite bin/yadm
19 cygpath -u "$1"
20 ^~~~~~~
21/nix/store/xhps4gff69lwdrmm0br3wcdmcw76lz1b-yadm-3.1.0/bin/yadm:2139: Couldn't resolve command 'cygpath'
22error: builder for '/nix/store/n7m489dav4fi4pwpvsfk0xzk8745782y-yadm-3.1.0.drv' failed with exit code 3
As we just saw, the typical response
when resholve reports a missing command is
to just add a package that includes it to
the inputs.
But this isn't always possible.
In this case, cygpath is from cygwin. I don't
think we have it in nixpkgs, so the best
approach is to use a fake
directive to
tell resholve to pretend something is defined
(when there's a really good reason
it's missing).
Other examples of when this approach
is useful:
- Fake a function that a Shell library
calls but does not define (because it
is a hook/callback that sourcing scripts
are supposed to fefine).
- Fake a builtin for some other shell (or
if you're using loadable builtins and
such.)
In this case, I'll only fake cygpath when
the stdenv is not cygwin:
1fake = {
2 external = if stdenv.isCygwin then [ ] else [ "cygpath" ];
3};
This will hopefully let cygpath break
naturally if someone does try to build
this package on cygwin, and then figure out the
full / correct fix when that happens.
build_after.console
1bash-5.1 $ nix-build yadm_after.nix --out-link result_after
2/nix/store/g424p1fcphah2jmc1yq5hsv4rmp3nzyf-yadm-3.1.0
If we get to the other side of the conversion
process, the dependencies we were able to
specify should be nailed down. For reference,
here's a before/after diff illustrating what
resholve did to this script:
resolved YADM
1bash-5.1 $ git diff --no-index result_before/bin/yadm result_after/bin/yadm
2diff --git a/result_before/bin/yadm b/result_after/bin/yadm
3index 9f57f60..f8340d8 100755
4--- a/result_before/bin/yadm
5+++ b/result_after/bin/yadm
6@@ -18,7 +18,7 @@
7 # shellcheck shell=bash
8 # execute script with bash (shebang line is /bin/sh for portability)
9 if [ -z "$BASH_VERSION" ]; then
10- [ "$YADM_TEST" != 1 ] && exec bash "$0" "$@"
11+ [ "$YADM_TEST" != 1 ] && exec /nix/store/pcjan45rssdn01cxx3sjg70avjg6c3ni-bash-4.4-p23/bin/bash "$0" "$@"
12 fi
13
14 VERSION=3.1.0
15@@ -94,8 +94,8 @@ function main() {
16 FULL_COMMAND="${_fc[*]}"
17
18 # create the YADM_DIR & YADM_DATA if they doesn't exist yet
19- [ -d "$YADM_DIR" ] || mkdir -p "$YADM_DIR"
20- [ -d "$YADM_DATA" ] || mkdir -p "$YADM_DATA"
21+ [ -d "$YADM_DIR" ] || /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mkdir -p "$YADM_DIR"
22+ [ -d "$YADM_DATA" ] || /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mkdir -p "$YADM_DATA"
23
24 # parse command line arguments
25 local retval=0
26@@ -422,16 +422,16 @@ function conditions() {
27 }
28 EOF
29
30- "${AWK_PROGRAM[0]}" \
31+ "/nix/store/8kpxw5na07ggdl2bs8kiwysif7120r6g-gawk-5.1.0/bin/awk" \
32 -v class="$local_class" \
33 -v os="$local_system" \
34 -v host="$local_host" \
35 -v user="$local_user" \
36 -v distro="$local_distro" \
37 -v source="$input" \
38- -v source_dir="$(dirname "$input")" \
39+ -v source_dir="$(/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/dirname "$input")" \
40 "$awk_pgm" \
41- "$input" > "$temp_file" || rm -f "$temp_file"
42+ "$input" > "$temp_file" || /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/rm -f "$temp_file"
43
44 move_file "$input" "$output" "$temp_file"
45 }
46@@ -447,7 +447,7 @@ function template_j2cli() {
47 YADM_USER="$local_user" \
48 YADM_DISTRO="$local_distro" \
49 YADM_SOURCE="$input" \
50- "$J2CLI_PROGRAM" "$input" -o "$temp_file"
51+ "/nix/store/p0876isrv4gi1pbfkc6bms737grzwjlj-python3.8-j2cli-0.3.10/bin/j2" "$input" -o "$temp_file"
52
53 move_file "$input" "$output" "$temp_file"
54 }
55@@ -473,7 +473,7 @@ function template_esh() {
56 output="$2"
57 temp_file="${output}.$$.$RANDOM"
58
59- "$ESH_PROGRAM" -o "$temp_file" "$input" \
60+ "/nix/store/irnmflhx17rlhzvz6g6d3f788aykf571-esh-0.1.1/bin/esh" -o "$temp_file" "$input" \
61 YADM_CLASS="$local_class" \
62 YADM_OS="$local_system" \
63 YADM_HOSTNAME="$local_host" \
64@@ -494,9 +494,9 @@ function move_file() {
65 # if the output files already exists as read-only, change it to be writable.
66 # there are some environments in which a read-only file will prevent the move
67 # from being successful.
68- [[ -e "$output" && ! -w "$output" ]] && chmod u+w "$output"
69+ [[ -e "$output" && ! -w "$output" ]] && /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/chmod u+w "$output"
70
71- mv -f "$temp_file" "$output"
72+ /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mv -f "$temp_file" "$output"
73 copy_perms "$input" "$output"
74 }
75
76@@ -528,7 +528,7 @@ function alt() {
77 # determine all tracked files
78 local tracked_files=()
79 local IFS=$'\n'
80- for tracked_file in $("$GIT_PROGRAM" ls-files | LC_ALL=C sort); do
81+ for tracked_file in $("/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" ls-files | LC_ALL=C /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/sort); do
82 tracked_files+=("$tracked_file")
83 done
84
85@@ -593,12 +593,12 @@ function remove_stale_links() {
86 if readlink_available; then
87 for stale_candidate in "${possible_alts[@]}"; do
88 if [ -L "$stale_candidate" ]; then
89- src=$(readlink "$stale_candidate" 2>/dev/null)
90+ src=$(/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/readlink "$stale_candidate" 2>/dev/null)
91 if [ -n "$src" ]; then
92 for review_link in "${alt_linked[@]}"; do
93 [ "$src" = "$review_link" ] && continue 2
94 done
95- rm -f "$stale_candidate"
96+ /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/rm -f "$stale_candidate"
97 fi
98 fi
99 done
100@@ -616,13 +616,13 @@ function set_local_alt_values() {
101
102 local_host="$(config local.hostname)"
103 if [ -z "$local_host" ] ; then
104- local_host=$(uname -n)
105+ local_host=$(/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/uname -n)
106 local_host=${local_host%%.*} # trim any domain from hostname
107 fi
108
109 local_user="$(config local.user)"
110 if [ -z "$local_user" ] ; then
111- local_user=$(id -u -n)
112+ local_user=$(/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/id -u -n)
113 fi
114
115 local_distro="$(query_distro)"
116@@ -636,7 +636,7 @@ function alt_linking() {
117 local alt_sources=()
118 local alt_template_cmds=()
119
120- for alt_path in $(for tracked in "${tracked_files[@]}"; do printf "%s\n" "$tracked" "${tracked%/*}"; done | LC_ALL=C sort -u) "${ENCRYPT_INCLUDE_FILES[@]}"; do
121+ for alt_path in $(for tracked in "${tracked_files[@]}"; do printf "%s\n" "$tracked" "${tracked%/*}"; done | LC_ALL=C /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/sort -u) "${ENCRYPT_INCLUDE_FILES[@]}"; do
122 alt_path="$YADM_BASE/$alt_path"
123 if [[ "$alt_path" =~ .\#\#. ]]; then
124 if [ -e "$alt_path" ] ; then
125@@ -656,7 +656,7 @@ function alt_linking() {
126 # ensure the destination path exists
127 assert_parent "$tgt"
128 # remove any existing symlink before processing template
129- [ -L "$tgt" ] && rm -f "$tgt"
130+ [ -L "$tgt" ] && /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/rm -f "$tgt"
131 "$template_cmd" "$src" "$tgt"
132 elif [ -n "$src" ]; then
133 # a link source is defined, create symlink
134@@ -666,8 +666,8 @@ function alt_linking() {
135 assert_parent "$tgt"
136 if [ "$do_copy" -eq 1 ]; then
137 # remove any existing symlink before copying
138- [ -L "$tgt" ] && rm -f "$tgt"
139- cp -f "$src" "$tgt"
140+ [ -L "$tgt" ] && /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/rm -f "$tgt"
141+ /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/cp -f "$src" "$tgt"
142 else
143 ln_relative "$src" "$tgt"
144 fi
145@@ -686,7 +686,7 @@ function ln_relative() {
146 fi
147 local rel_source
148 rel_source=$(relative_path "$target_dir" "$full_source")
149- ln -nfs "$rel_source" "$full_target"
150+ /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/ln -nfs "$rel_source" "$full_target"
151 alt_linked+=("$rel_source")
152 }
153
154@@ -750,8 +750,8 @@ function clone() {
155 # remove existing if forcing the clone to happen anyway
156 [ -d "$YADM_REPO" ] && {
157 debug "Removing existing repo prior to clone"
158- "$GIT_PROGRAM" -C "$YADM_WORK" submodule deinit -f --all
159- rm -rf "$YADM_REPO"
160+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" -C "$YADM_WORK" submodule deinit -f --all
161+ /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/rm -rf "$YADM_REPO"
162 }
163
164 local wc
165@@ -761,22 +761,22 @@ function clone() {
166 # first clone without checkout
167 debug "Doing an initial clone of the repository"
168 (cd "$wc" &&
169- "$GIT_PROGRAM" -c core.sharedrepository=0600 clone --no-checkout \
170+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" -c core.sharedrepository=0600 clone --no-checkout \
171 --separate-git-dir="$YADM_REPO" "${args[@]}" repo.git) || {
172 debug "Removing repo after failed clone"
173- rm -rf "$YADM_REPO" "$wc"
174+ /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/rm -rf "$YADM_REPO" "$wc"
175 error_out "Unable to clone the repository"
176 }
177- rm -rf "$wc"
178+ /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/rm -rf "$wc"
179 configure_repo
180
181 # then reset the index as the --no-checkout flag makes the index empty
182- "$GIT_PROGRAM" reset --quiet -- .
183+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" reset --quiet -- .
184
185 if [ "$YADM_WORK" = "$HOME" ]; then
186 debug "Determining if repo tracks private directories"
187 for private_dir in $(private_dirs all); do
188- found_log=$("$GIT_PROGRAM" log -n 1 -- "$private_dir" 2>/dev/null)
189+ found_log=$("/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" log -n 1 -- "$private_dir" 2>/dev/null)
190 if [ -n "$found_log" ]; then
191 debug "Private directory $private_dir is tracked by repo"
192 assert_private_dirs "$private_dir"
193@@ -790,11 +790,11 @@ function clone() {
194
195 cd_work "Clone" || return
196
197- "$GIT_PROGRAM" ls-files --deleted | while IFS= read -r file; do
198- "$GIT_PROGRAM" checkout -- ":/$file"
199+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" ls-files --deleted | while IFS= read -r file; do
200+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" checkout -- ":/$file"
201 done
202
203- if [ -n "$("$GIT_PROGRAM" ls-files --modified)" ]; then
204+ if [ -n "$("/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" ls-files --modified)" ]; then
205 local msg
206 IFS='' read -r -d '' msg <<EOF
207 **NOTE**
208@@ -846,7 +846,7 @@ EOF
209
210 # operate on the yadm repo's configuration file
211 # this is always local to the machine
212- "$GIT_PROGRAM" config "$@"
213+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" config "$@"
214
215 CHANGES_POSSIBLE=1
216
217@@ -854,7 +854,7 @@ EOF
218 # make sure parent folder of config file exists
219 assert_parent "$YADM_CONFIG"
220 # operate on the yadm configuration file
221- "$GIT_PROGRAM" config --file="$(mixed_path "$YADM_CONFIG")" "$@"
222+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" config --file="$(mixed_path "$YADM_CONFIG")" "$@"
223
224 fi
225
226@@ -906,13 +906,13 @@ function _decrypt_from() {
227 case "$yadm_cipher" in
228 gpg)
229 require_gpg
230- $GPG_PROGRAM -d "$output_archive"
231+ /nix/store/y2hcv73jwa5k1s1dhj65mjz4cwyrfzm8-gnupg-2.2.27/bin/gpg -d "$output_archive"
232 ;;
233
234 openssl)
235 require_openssl
236 _set_openssl_options
237- $OPENSSL_PROGRAM enc -d "${OPENSSL_OPTS[@]}" -in "$output_archive"
238+ /nix/store/gdrl09cxsbml1gdb0dqkjb11i1s36183-openssl-1.1.1k-bin/bin/openssl enc -d "${OPENSSL_OPTS[@]}" -in "$output_archive"
239 ;;
240
241 *)
242@@ -933,13 +933,13 @@ function _encrypt_to() {
243 gpg)
244 require_gpg
245 _set_gpg_options
246- $GPG_PROGRAM --yes "${GPG_OPTS[@]}" --output "$output_archive"
247+ /nix/store/y2hcv73jwa5k1s1dhj65mjz4cwyrfzm8-gnupg-2.2.27/bin/gpg --yes "${GPG_OPTS[@]}" --output "$output_archive"
248 ;;
249
250 openssl)
251 require_openssl
252 _set_openssl_options
253- $OPENSSL_PROGRAM enc -e "${OPENSSL_OPTS[@]}" -out "$output_archive"
254+ /nix/store/gdrl09cxsbml1gdb0dqkjb11i1s36183-openssl-1.1.1k-bin/bin/openssl enc -e "${OPENSSL_OPTS[@]}" -out "$output_archive"
255 ;;
256
257 *)
258@@ -963,7 +963,7 @@ function decrypt() {
259 fi
260
261 # decrypt the archive
262- if (_decrypt_from "$YADM_ARCHIVE" || echo 1) | tar v${tar_option}f - -C "$YADM_WORK"; then
263+ if (_decrypt_from "$YADM_ARCHIVE" || echo 1) | /nix/store/gh440dlrbabj33lj410w37qpjfidhgnd-gnutar-1.34/bin/tar v${tar_option}f - -C "$YADM_WORK"; then
264 [ ! "$DO_LIST" = "YES" ] && echo "All files decrypted."
265 else
266 error_out "Unable to extract encrypted files."
267@@ -987,21 +987,21 @@ function encrypt() {
268 echo
269
270 # encrypt all files which match the globs
271- if tar -f - -c "${ENCRYPT_INCLUDE_FILES[@]}" | _encrypt_to "$YADM_ARCHIVE"; then
272+ if /nix/store/gh440dlrbabj33lj410w37qpjfidhgnd-gnutar-1.34/bin/tar -f - -c "${ENCRYPT_INCLUDE_FILES[@]}" | _encrypt_to "$YADM_ARCHIVE"; then
273 echo "Wrote new file: $YADM_ARCHIVE"
274 else
275 error_out "Unable to write $YADM_ARCHIVE"
276 fi
277
278 # offer to add YADM_ARCHIVE if untracked
279- archive_status=$("$GIT_PROGRAM" status --porcelain -uall "$(mixed_path "$YADM_ARCHIVE")" 2>/dev/null)
280+ archive_status=$("/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" status --porcelain -uall "$(mixed_path "$YADM_ARCHIVE")" 2>/dev/null)
281 archive_regex="^\?\?"
282 if [[ $archive_status =~ $archive_regex ]] ; then
283 echo "It appears that $YADM_ARCHIVE is not tracked by yadm's repository."
284 echo "Would you like to add it now? (y/n)"
285 read -r answer < /dev/tty
286 if [[ $answer =~ ^[yY]$ ]] ; then
287- "$GIT_PROGRAM" add "$(mixed_path "$YADM_ARCHIVE")"
288+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" add "$(mixed_path "$YADM_ARCHIVE")"
289 fi
290 fi
291
292@@ -1083,8 +1083,8 @@ function git_command() {
293 CHANGES_POSSIBLE=1
294
295 # pass commands through to git
296- debug "Running git command $GIT_PROGRAM $*"
297- "$GIT_PROGRAM" "$@"
298+ debug "Running git command git $*"
299+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" "$@"
300 return "$?"
301 }
302
303@@ -1139,13 +1139,13 @@ function init() {
304 # remove existing if forcing the init to happen anyway
305 [ -d "$YADM_REPO" ] && {
306 debug "Removing existing repo prior to init"
307- "$GIT_PROGRAM" -C "$YADM_WORK" submodule deinit -f --all
308- rm -rf "$YADM_REPO"
309+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" -C "$YADM_WORK" submodule deinit -f --all
310+ /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/rm -rf "$YADM_REPO"
311 }
312
313 # init a new bare repo
314 debug "Init new repo"
315- "$GIT_PROGRAM" init --shared=0600 --bare "$(mixed_path "$YADM_REPO")" "$@"
316+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" init --shared=0600 --bare "$(mixed_path "$YADM_REPO")" "$@"
317 configure_repo
318
319 CHANGES_POSSIBLE=1
320@@ -1239,7 +1239,7 @@ function list() {
321 fi
322
323 # list tracked files
324- "$GIT_PROGRAM" ls-files
325+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" ls-files
326
327 }
328
329@@ -1276,7 +1276,7 @@ function perms() {
330 # remove group/other permissions from collected globs
331 #shellcheck disable=SC2068
332 #(SC2068 is disabled because in this case, we desire globbing)
333- chmod -f go-rwx ${GLOBS[@]} &> /dev/null
334+ /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/chmod -f go-rwx ${GLOBS[@]} &> /dev/null
335 # TODO: detect and report changing permissions in a portable way
336
337 }
338@@ -1309,18 +1309,18 @@ function upgrade() {
339
340 # Must absorb git dirs, otherwise deinit below will fail for modules that have
341 # been cloned first and then added as a submodule.
342- "$GIT_PROGRAM" submodule absorbgitdirs
343+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" submodule absorbgitdirs
344
345 local submodule_status
346- submodule_status=$("$GIT_PROGRAM" -C "$YADM_WORK" submodule status)
347+ submodule_status=$("/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" -C "$YADM_WORK" submodule status)
348 while read -r sha submodule rest; do
349 [ "$submodule" == "" ] && continue
350 if [[ "$sha" = -* ]]; then
351 continue
352 fi
353- "$GIT_PROGRAM" -C "$YADM_WORK" submodule deinit ${FORCE:+-f} -- "$submodule" || {
354+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" -C "$YADM_WORK" submodule deinit ${FORCE:+-f} -- "$submodule" || {
355 for other in "${submodules[@]}"; do
356- "$GIT_PROGRAM" -C "$YADM_WORK" submodule update --init --recursive -- "$other"
357+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" -C "$YADM_WORK" submodule update --init --recursive -- "$other"
358 done
359 error_out "Unable to upgrade. Could not deinit submodule $submodule"
360 }
361@@ -1328,7 +1328,7 @@ function upgrade() {
362 done <<< "$submodule_status"
363
364 assert_parent "$YADM_REPO"
365- mv "$LEGACY_REPO" "$YADM_REPO"
366+ /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mv "$LEGACY_REPO" "$YADM_REPO"
367 fi
368 fi
369 GIT_DIR="$YADM_REPO"
370@@ -1345,10 +1345,10 @@ function upgrade() {
371 echo "Moving $LEGACY_ARCHIVE to $YADM_ARCHIVE"
372 assert_parent "$YADM_ARCHIVE"
373 # test to see if path is "tracked" in repo, if so 'git mv' must be used
374- if "$GIT_PROGRAM" ls-files --error-unmatch "$LEGACY_ARCHIVE" &> /dev/null; then
375- "$GIT_PROGRAM" mv "$LEGACY_ARCHIVE" "$YADM_ARCHIVE" && repo_updates=1
376+ if "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" ls-files --error-unmatch "$LEGACY_ARCHIVE" &> /dev/null; then
377+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" mv "$LEGACY_ARCHIVE" "$YADM_ARCHIVE" && repo_updates=1
378 else
379- mv -i "$LEGACY_ARCHIVE" "$YADM_ARCHIVE"
380+ /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mv -i "$LEGACY_ARCHIVE" "$YADM_ARCHIVE"
381 fi
382 fi
383
384@@ -1367,17 +1367,17 @@ function upgrade() {
385 echo "Moving $legacy_path to $new_filename"
386 assert_parent "$new_filename"
387 # test to see if path is "tracked" in repo, if so 'git mv' must be used
388- if "$GIT_PROGRAM" ls-files --error-unmatch "$legacy_path" &> /dev/null; then
389- "$GIT_PROGRAM" mv "$legacy_path" "$new_filename" && repo_updates=1
390+ if "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" ls-files --error-unmatch "$legacy_path" &> /dev/null; then
391+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" mv "$legacy_path" "$new_filename" && repo_updates=1
392 else
393- mv -i "$legacy_path" "$new_filename"
394+ /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mv -i "$legacy_path" "$new_filename"
395 fi
396 fi
397 done
398
399 # handle submodules, which need to be reinitialized
400 for submodule in "${submodules[@]}"; do
401- "$GIT_PROGRAM" -C "$YADM_WORK" submodule update --init --recursive -- "$submodule"
402+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" -C "$YADM_WORK" submodule update --init --recursive -- "$submodule"
403 done
404
405 [ "$actions_performed" -eq 0 ] && \
406@@ -1647,7 +1647,7 @@ function configure_paths() {
407 # obtain YADM_WORK from repo if it exists
408 if [ -d "$GIT_DIR" ]; then
409 local work
410- work=$(unix_path "$("$GIT_PROGRAM" config core.worktree)")
411+ work=$(unix_path "$("/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" config core.worktree)")
412 [ -n "$work" ] && YADM_WORK="$work"
413 fi
414
415@@ -1666,16 +1666,16 @@ function configure_repo() {
416 debug "Configuring new repo"
417
418 # change bare to false (there is a working directory)
419- "$GIT_PROGRAM" config core.bare 'false'
420+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" config core.bare 'false'
421
422 # set the worktree for the yadm repo
423- "$GIT_PROGRAM" config core.worktree "$(mixed_path "$YADM_WORK")"
424+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" config core.worktree "$(mixed_path "$YADM_WORK")"
425
426 # by default, do not show untracked files and directories
427- "$GIT_PROGRAM" config status.showUntrackedFiles no
428+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" config status.showUntrackedFiles no
429
430 # possibly used later to ensure we're working on the yadm repo
431- "$GIT_PROGRAM" config yadm.managed 'true'
432+ "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" config yadm.managed 'true'
433
434 }
435
436@@ -1684,16 +1684,16 @@ function set_operating_system() {
437 if [[ "$(<$PROC_VERSION)" =~ [Mm]icrosoft ]]; then
438 OPERATING_SYSTEM="WSL"
439 else
440- OPERATING_SYSTEM=$(uname -s)
441+ OPERATING_SYSTEM=$(/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/uname -s)
442 fi 2>/dev/null
443
444 case "$OPERATING_SYSTEM" in
445 CYGWIN*|MINGW*|MSYS*)
446- git_version="$("$GIT_PROGRAM" --version 2>/dev/null)"
447+ git_version="$("/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" --version 2>/dev/null)"
448 if [[ "$git_version" =~ windows ]] ; then
449 USE_CYGPATH=1
450 fi
451- OPERATING_SYSTEM=$(uname -o)
452+ OPERATING_SYSTEM=$(/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/uname -o)
453 ;;
454 *)
455 ;;
456@@ -1703,7 +1703,7 @@ function set_operating_system() {
457
458 function set_awk() {
459 local pgm
460- for pgm in "${AWK_PROGRAM[@]}"; do
461+ for pgm in "awk"; do
462 command -v "$pgm" &> /dev/null && AWK_PROGRAM=("$pgm") && return
463 done
464 }
465@@ -1800,7 +1800,7 @@ function assert_private_dirs() {
466 if [ ! -d "$YADM_WORK/$private_dir" ]; then
467 debug "Creating $YADM_WORK/$private_dir"
468 #shellcheck disable=SC2174
469- mkdir -m 0700 -p "$YADM_WORK/$private_dir" &> /dev/null
470+ /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mkdir -m 0700 -p "$YADM_WORK/$private_dir" &> /dev/null
471 fi
472 done
473 }
474@@ -1808,7 +1808,7 @@ function assert_private_dirs() {
475 function assert_parent() {
476 basedir=${1%/*}
477 if [ -n "$basedir" ]; then
478- [ -e "$basedir" ] || mkdir -p "$basedir"
479+ [ -e "$basedir" ] || /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mkdir -p "$basedir"
480 fi
481 }
482
483@@ -1816,7 +1816,7 @@ function display_private_perms() {
484 when="$1"
485 for private_dir in $(private_dirs all); do
486 if [ -d "$YADM_WORK/$private_dir" ]; then
487- private_perms=$(ls -ld "$YADM_WORK/$private_dir")
488+ private_perms=$(/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/ls -ld "$YADM_WORK/$private_dir")
489 debug "$when" private dir perms "$private_perms"
490 fi
491 done
492@@ -1890,7 +1890,7 @@ function parse_encrypt() {
493
494 # sort the encrypted files
495 #shellcheck disable=SC2207
496- IFS=$'\n' ENCRYPT_INCLUDE_FILES=($(LC_ALL=C sort <<<"${FINAL_INCLUDE[*]}"))
497+ IFS=$'\n' ENCRYPT_INCLUDE_FILES=($(LC_ALL=C /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/sort <<<"${FINAL_INCLUDE[*]}"))
498 unset IFS
499
500 if [ "$unset_globstar" = "1" ]; then
501@@ -2016,10 +2016,10 @@ function get_mode {
502 local mode
503
504 # most *nixes
505- mode=$(stat -c '%a' "$filename" 2>/dev/null)
506+ mode=$(/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/stat -c '%a' "$filename" 2>/dev/null)
507 if [ -z "$mode" ] ; then
508 # BSD-style
509- mode=$(stat -f '%p' "$filename" 2>/dev/null)
510+ mode=$(/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/stat -f '%p' "$filename" 2>/dev/null)
511 mode=${mode: -4}
512 fi
513
514@@ -2035,7 +2035,7 @@ function copy_perms {
515 local source="$1"
516 local dest="$2"
517 mode=$(get_mode "$source")
518- [ -n "$mode" ] && chmod "$mode" "$dest"
519+ [ -n "$mode" ] && /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/chmod "$mode" "$dest"
520 return 0
521 }
522
523@@ -2063,8 +2063,8 @@ function require_git() {
524 GIT_PROGRAM="$alt_git"
525 more_info="\nThis command has been set via the yadm.git-program configuration."
526 fi
527- command -v "$GIT_PROGRAM" &> /dev/null ||
528- error_out "This functionality requires Git to be installed, but the command '$GIT_PROGRAM' cannot be located.$more_info"
529+ command -v "/nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git" &> /dev/null ||
530+ error_out "This functionality requires Git to be installed, but the command 'git' cannot be located.$more_info"
531 }
532 function require_gpg() {
533 local alt_gpg
534@@ -2076,8 +2076,8 @@ function require_gpg() {
535 GPG_PROGRAM="$alt_gpg"
536 more_info="\nThis command has been set via the yadm.gpg-program configuration."
537 fi
538- command -v "$GPG_PROGRAM" &> /dev/null ||
539- error_out "This functionality requires GPG to be installed, but the command '$GPG_PROGRAM' cannot be located.$more_info"
540+ command -v "/nix/store/y2hcv73jwa5k1s1dhj65mjz4cwyrfzm8-gnupg-2.2.27/bin/gpg" &> /dev/null ||
541+ error_out "This functionality requires GPG to be installed, but the command 'gpg' cannot be located.$more_info"
542 }
543 function require_openssl() {
544 local alt_openssl
545@@ -2089,8 +2089,8 @@ function require_openssl() {
546 OPENSSL_PROGRAM="$alt_openssl"
547 more_info="\nThis command has been set via the yadm.openssl-program configuration."
548 fi
549- command -v "$OPENSSL_PROGRAM" &> /dev/null ||
550- error_out "This functionality requires OpenSSL to be installed, but the command '$OPENSSL_PROGRAM' cannot be located.$more_info"
551+ command -v "/nix/store/gdrl09cxsbml1gdb0dqkjb11i1s36183-openssl-1.1.1k-bin/bin/openssl" &> /dev/null ||
552+ error_out "This functionality requires OpenSSL to be installed, but the command 'openssl' cannot be located.$more_info"
553 }
554 function require_repo() {
555 [ -d "$YADM_REPO" ] || error_out "Git repo does not exist. did you forget to run 'init' or 'clone'?"
556@@ -2111,11 +2111,11 @@ function bootstrap_available() {
557 return 1
558 }
559 function awk_available() {
560- command -v "${AWK_PROGRAM[0]}" &> /dev/null && return
561+ command -v "/nix/store/8kpxw5na07ggdl2bs8kiwysif7120r6g-gawk-5.1.0/bin/awk" &> /dev/null && return
562 return 1
563 }
564 function j2cli_available() {
565- command -v "$J2CLI_PROGRAM" &> /dev/null && return
566+ command -v "/nix/store/p0876isrv4gi1pbfkc6bms737grzwjlj-python3.8-j2cli-0.3.10/bin/j2" &> /dev/null && return
567 return 1
568 }
569 function envtpl_available() {
570@@ -2123,11 +2123,11 @@ function envtpl_available() {
571 return 1
572 }
573 function esh_available() {
574- command -v "$ESH_PROGRAM" &> /dev/null && return
575+ command -v "/nix/store/irnmflhx17rlhzvz6g6d3f788aykf571-esh-0.1.1/bin/esh" &> /dev/null && return
576 return 1
577 }
578 function readlink_available() {
579- command -v "readlink" &> /dev/null && return
580+ command -v "/nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/readlink" &> /dev/null && return
581 return 1
582 }
583
584@@ -2175,3 +2175,41 @@ if [ "$YADM_TEST" != 1 ] ; then
585 configure_paths
586 main "${MAIN_ARGS[@]}"
587 fi
588+
589+### resholve directives (auto-generated) ## format_version: 2
590+# resholve: fake external:cygpath
591+# resholve: fix $AWK_PROGRAM:awk
592+# resholve: fix $ESH_PROGRAM:esh
593+# resholve: fix $GIT_PROGRAM:git
594+# resholve: fix $GPG_PROGRAM:gpg
595+# resholve: fix $J2CLI_PROGRAM:j2
596+# resholve: fix $OPENSSL_PROGRAM:openssl
597+# resholve: keep $ENVTPL_PROGRAM
598+# resholve: keep $LSB_RELEASE_PROGRAM
599+# resholve: keep $SHELL
600+# resholve: keep $YADM_COMMAND
601+# resholve: keep $hook_command
602+# resholve: keep $template_cmd
603+# resholve: keep /nix/store/8kpxw5na07ggdl2bs8kiwysif7120r6g-gawk-5.1.0/bin/awk
604+# resholve: keep /nix/store/g7da36vi3b6ybqbld2jzlz2575mgamfq-git-2.31.1/bin/git
605+# resholve: keep /nix/store/gdrl09cxsbml1gdb0dqkjb11i1s36183-openssl-1.1.1k-bin/bin/openssl
606+# resholve: keep /nix/store/gh440dlrbabj33lj410w37qpjfidhgnd-gnutar-1.34/bin/tar
607+# resholve: keep /nix/store/irnmflhx17rlhzvz6g6d3f788aykf571-esh-0.1.1/bin/esh
608+# resholve: keep /nix/store/p0876isrv4gi1pbfkc6bms737grzwjlj-python3.8-j2cli-0.3.10/bin/j2
609+# resholve: keep /nix/store/pcjan45rssdn01cxx3sjg70avjg6c3ni-bash-4.4-p23/bin/bash
610+# resholve: keep /nix/store/y2hcv73jwa5k1s1dhj65mjz4cwyrfzm8-gnupg-2.2.27/bin/gpg
611+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/chmod
612+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/cp
613+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/dirname
614+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/id
615+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/ln
616+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/ls
617+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mkdir
618+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/mv
619+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/readlink
620+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/rm
621+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/sort
622+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/stat
623+# resholve: keep /nix/store/yzvj23zkg314xjywc3dmzdlqchkqq4m0-coreutils-8.32/bin/uname
624+# resholve: keep exec:$YADM_BOOTSTRAP
625+