Nix Attribute Set (attrset) Laziness
Great article, minor nitpick: We usually call them attribute sets (or attrsets), to differentiate from normal (multi-)sets (collections of values with no identifiers except maybe equality). They are more akin to (hash)maps (and internally probably implemented as such).
— Patschisupercalifragilisticexpialidociousdöner (@Profpatsch) December 31, 2017
An erronous statement such as
concat = builtins.nada ""
where the attribute nada
is undefined, will not be evaluated immediately
after definition. Nix is lazy enough when it comes to attrsets to wait until
the call to concat
to figure out the contents of the set and attempt to invoke
the function.
Conversely, something like
concat = nada ""
would fail at definition as nada
is clearly undefined. The laziness of the
attrset is no longer there to defer the evaluation of the right-hand side of
this expression.
Note how
z = let
concat = builtins.nada " ";
items = ["leni" "photo"];
in {
out = "${concat items}";
} // { out = "wow"; }
represents the definition of an attrset, which will be evaluated lazily.
Furthermore the the builtins.nada
phrase will also be evaluated lazily. As
such, the first call of z
, evaluates all attributes of the overriden attrset
to return an attrset resembling { out = "wow"; }
. This output suggests that
the erronous call was completely avoided as a result of the override of the
out
attribute during merge (the //
operator merges two attrsets).
I guess in the case of merges, one may assume that an attribute in the left-hand side is optimized into oblivion if a similarly-named attribute is present on the right-hand side as the right-hand side takes precedence.
For a recursive attrset
z = let
concat = builtins.nada " ";
items = ["leni" "photo"];
in rec {
a = "${concat items}";
} // { a = "wow"; }
the same holds true until a reference is made to the defective attribute as in
z = let
concat = builtins.nada " ";
items = ["leni" "photo"];
in rec {
a = "${concat items}";
b = a; # <- reference to broken attribute
} // { a = "wow"; }
which fails comically with a fragmented error message :stuck_out_tongue_closed_eyes:
{ a = "wow"; b = error: attribute ‘nada’ missing, at (string):2:12
which could leave one to assume that recursive attrsets have to be considered
a bit less lazy when recursive references are made. It seems that a
was
only overriden after all references to a
within the left-hand operand were
evaluated. This is kind of a big deal, as that demands a bit more care from
anyone using recursive attrsets with the intent to override them later. :boom:
Fixing the concat
partial by, for example definining it to
concat = builtins.concatStringsSep " "
results to the expression
z = let
concat = builtins.concatStringsSep " ";
items = ["leni" "photo"];
in rec {
a = "${concat items}";
b = a; # <- no longer broken
} // { a = "wow"; }
producing { a = "wow"; b = "leni photo"; }
which more or less demonstrates
that the solving of the left-hand operand was forced since the left-hand
attribute b
referenced attribute a
within its containing attrset. In that
case the merge seems to be performed after the left-hand operand is “solved”.
When we abstain from referencing attribute a
in the left-hand operand as in
z = let
concat = builtins.nada " ";
items = ["leni" "photo"];
in rec {
a = "${concat items}";
b = "not referencing broken code";
} // { a = "wow"; }
we end up with the result
{ a = "wow"; b = "not referencing broken code"; }
which leads me to the following take-away:
The following pseudo-code demonstrates this problem in something remarkably stupid I attempted earlier on.
q = rec {
name = "some-package";
version = "0.1.0";
src = "old source";
installPhase = ''
#echo do something with ${src}
'';
} // {
src = "new source";
# I would have to redefine installPhase here or
# face the consequences of installPhase being
# evaluated against the old value of src
}
Note that the resulting attrset contains the updated src
but an installPhase
string that was evaluated against the old value of src
. :boom:
This broken code, I wrapped into a helper to compose a derivation
mkSomePackage = overrides: stdenv.mkDerivation (rec {
name = "some-package";
version = "0.1.0";
src = "some source";
installPhase = ''
#echo do something with ${src}
'';
} // overrides)
which resulted to the resulting derivation having some properly overriden attributes and some attributes evaluated against the original attributes (i.e.: being from the left-hand attrset) contributing to some confusion. :sob:
Future me reading this… you’ve been warned. :rage: