Glad it’s useful, I was trying to get the whole reasoning/process. Firehose v2 follows 
How did you find this URL?
Yes, nix-shell (and nix-env with nix-env -f <url>, as well as nix-build/nix build) support URLs, specifically tarballs, which is indeed pretty handy.
The URL is just a tar.gz archive of your github repo, so you do
and change the extension from
.zip to
.tar.gz (which GitHub supports, presumably this is the bit you didn’t know, I think they just use zip by default to avoid confusing non-Unix people).
Bracket notation in the build phrase list?
This will be an introduction to programming in sh/bash, for more details I refer you to the bash manual page, e.g. type in man bash into your terminal.
Bash is a slightly odd programming language, in that everything is quoted as a string, unless you use $ to get the value of that variable, e.g. foo is just a string just like "foo", but $foo is the value of the variable. Note, that $ expands variables inside double-quoted strings, but not single-quoted strings, so "$foo" is the value of the variable foo, while '$foo' is the string dollar foo.
In some cases, you want the value of the variable foo followed immediately by a string, say bar, so you use the curly brackets/braces, e.g. ${foo}bar, as $foobar is the value of the variable called foobar.
On top of this, bash supports some replacements, one of which is the “use this value if the variable is unset or empty”, which is the colon-minus ${prePhases:-} which says use the empty string if the variable prePhases is unset or empty.
Now, normally a variable that is unset is treated as the empty string, but for the sake of catching errors earlier, the builder is run with flags e/errexit (exit at errors) u/nounset (unset variables are an error), this is done by the file $stdenv/setup with set -eu.
Because of this, we use ${prePhases:-} instead of just ${prePhases}.
I ignored the bracketed expressions, because they are unset in your derivation. To set them, you could do in your default.nix something like
stdenv.mkDerivation rec {
... # Other stuff
prePhases = ''
do_the_thing
do_some_other_thing
'';
... # Other stuff
}
(the stdenv.mkDerivation is just for context as to where to put the prePhases).
I don’t see the patchPhase step doing anything. Is that normal?
Yes it’s normal for the patchPhase to do nothing, unless you have defined patches in your nix derivation.
But let me use this space for Bash tutorial part 2; what does ${patchPhase:-patchPhase} mean?
Well, it gets the value of the variable patchPhase, or if that is unset/empty, it uses the string patchPhase. But what is the string patchPhase used for then? Well, you can think of bash as having two stages, variable expansion, and evaluation. So once the command line has had variable expansion, say the variable patchPhase is unset/empty, we evaluate the command line patchPhase, for which bash searches for in the builtins, the defined functions, and the executable files in the colon-separated list of directories of the variable $PATH.
To see where it finds the patchPhase, you can use type patchPhase, or to see all matches, use type -a patchPhase. In this case it’s a function (that I know to be defined in the file $stdenv/setup.
How do I replace multiple lines?
Now, you might have the question, how do I stop/override a phase, or what went wrong with the patchPhase I tried?
Well, you were on the right track actually, you just need to tell bash that you are entering a multi-line expression, which is what the backslashes \ at the end of the line do. Also, you don’t need to escape a single-quote inside a double-quoted string. But this leads to having two single-quotes, which terminates the nix string. You can escape those with another single-quote. (Note, there is nix shell if you want to play around with nix expressions.)
patchPhase = ''
substituteInPlace wscript \
--replace "conf.env.GIT_COMMIT = subprocess.check_output(" \
"conf.env.GIT_COMMIT = 'cec0ea76b2d5f69afa74d3ffc14a0950e32a7914'" \
--replace "[\"git\", \"rev-parse\", \"--verify\", \"--short\", \"HEAD\"]," "" \
--replace "stderr=devnull).decode().strip()" "" \
--replace "conf.env.GIT_COMMIT = '''" \
"conf.env.GIT_COMMIT = 'cec0ea76b2d5f69afa74d3ffc14a0950e32a7914'"
'';
The backslashes need not be aligned, but it helps visually. They must however be the last thing on the line, not even space after them, as you are escaping the newline character.
As for disabling a phase, remember how I said the :- is for unset or empty, well there are nix variables for the mkDerivation such as dontConfigure or doCheck, but for the sake of some more bash, you could do something like configurePhase = ":" as in bash the colon (:) is the do nothing operation. Also, instead of :- for unset or empty, there is - for only unset, e.g. we could have used ${patchPhase-patchPhase} which when the environment variable patchPhase is empty, evaluates to empty.
As a side-note, yes, the bash variable expansion rules are unfortunately complex/subtle, and it’s one of those things that can lead to hard-to-find bugs.
As for how to do the substitution, I would just create a patch, say 0001-fix-wscript-lack-of-git.patch containing
--- a/wscript 2020-01-30 17:27:13.264537220 +0000
+++ b/wscript 2020-01-30 17:35:56.911563968 +0000
@@ -257,8 +257,9 @@
conf.env.GIT_COMMIT = subprocess.check_output(
["git", "rev-parse", "--verify", "--short", "HEAD"],
stderr=devnull).decode().strip()
- except subprocess.CalledProcessError:
- conf.env.GIT_COMMIT = ''
+ except (OSError, subprocess.CalledProcessError):
+ import os
+ conf.env.GIT_COMMIT = os.getenv("version")
conf.define("VERSION", VERSION)
conf.define("_GNU_SOURCE", 1)
Note, that tabs inevitably got lost here, where as the original wscript file had tabs, so you might have to recreate the patch on your machine. See here for patch creation details.
With the patch, it almost builds for me, except it fails for some pointer alignment reason
../third-party/libuv/src/unix/linux-core.c: In function 'uv__io_poll':
../third-party/libuv/src/unix/linux-core.c:268:5: error: converting a packed 'struct uv__epoll_event' pointer (alignment 1) to a 'uv__io_t' {aka 'struct uv__io_s'} pointer (alignment 8) may result in an unaligned pointer value [-Werror=address-of-packed-member]
268 | loop->watchers[loop->nwatchers] = (void*) events;
| ^~~~
In file included from ../third-party/libuv/src/unix/internal.h:37,
from ../third-party/libuv/src/unix/linux-core.c:22:
../third-party/libuv/src/unix/linux-syscalls.h:97:8: note: defined here
97 | struct uv__epoll_event {
| ^~~~~~~~~~~~~~~
In file included from /build/serialosc/third-party/libuv/include/uv.h:61,
from ../third-party/libuv/src/unix/linux-core.c:21:
/build/serialosc/third-party/libuv/include/uv-unix.h:84:8: note: defined here
84 | struct uv__io_s {
| ^~~~~~~~
cc1: all warnings being treated as errors
Waf: Leaving directory `/build/serialosc/build'
Build failed
-> task in 'LIBUV' failed (exit status 1):
{task 140737336483152: c linux-core.c -> linux-core.c.1.o}
['gcc', '-std=c99', '-Wall', '-Werror', '-pthread', '-g', '--std=gnu89', '-pedantic', '-Wall', '-Wextra', '-Wno-unused-parameter', '-Wstrict-aliasing', '-fPIC', '-Wno-cast-align', '-I/build/serialosc/build/third-party/libuv/include', '-I/build/serialosc/third-party/libuv/include', '-I/build/serialosc/build/third-party/libuv/src', '-I/build/serialosc/third-party/libuv/src', '-DHAVE_WORKING_POLL=1', '-DHAVE_LIBUDEV=1', '-DHAVE_LIBMONOME=1', '-DHAVE_LO=1', '-DHAVE_STRDUP=1', '-DHAVE_STRNDUP=1', '-DHAVE_STRCASECMP=1', '-DHAVE_UNISTD_H=1', '-DHAVE_DNS_SD_H=1', '-DVERSION="1.4.1"', '-D_GNU_SOURCE=1', '-DGIT_COMMIT="v1.4.1"', '-D_LARGEFILE_SOURCE', '-D_FILE_OFFSET_BITS=64', '-D_GNU_SOURCE', '-D_LARGEFILE_SOURCE', '-D_FILE_OFFSET_BITS=64', '-D_POSIX_C_SOURCE=200112', '../third-party/libuv/src/unix/linux-core.c', '-c', '-o', '/build/serialosc/build/third-party/libuv/src/unix/linux-core.c.1.o']
note: keeping build directory '/tmp/nix-build-serialosc.drv-2'
builder for '/nix/store/01g9ikl18plly7big7ajkih8kr0n5zgi-serialosc.drv' failed with exit code 1
error: build of '/nix/store/01g9ikl18plly7big7ajkih8kr0n5zgi-serialosc.drv' failed
You’d have to ask serialosc’s developers about this issue.
