Package bash script with .txt dependencies in NixOS

I wrote a simple bash script. Here’s the description:

"It has a list of banned words, it recursively searches the entire project in search of these banned words, and if it finds them it alerts you to exactly where."

I wrote some nixos config code to have an alias to this script so I could execute it anywhere.

(Nix Alias Code)
{ pkgs, ... }:
let
  check-names = { pkgs, ... }:
  pkgs.writeScriptBin "check-names" 
    (builtins.readFile ./CheckNames.sh);
in {
  environment.systemPackages = [
    # Add script as package
    (check-names pkgs)
  ];
}

The problem:

The bash script has a txt dependency named bannedwords.txt that isn’t carried up to the nix store, which is a crucial element of the script that cannot be embedded into the script itself (It’s a valuable set of words that can’t be shared)

Is there a way, in Nix and NixOS, to alias the script in a way so I could put both the bash script and the .txt file in the same directory?

If that’s impossible or a headache or just a stupid way of doing things, I’m open to suggestions on how I could improve the script.

(Bash Script Code)
# !/bin/bash

# How this script works: It has a list of banned words, it recursively searches the entire project
# in search of these banned words, and if it finds them it alerts you to exactly where.
#
# Put a bannedwords.txt in this directory
#
# Useful for keeping identities/secrets separate

path=${1:-$PWD} # If path supplied, use that. Otherwise use current terminal path

# Determine the directory where the script is located
script_dir=$(dirname "$(readlink -f "$0")")

# bannedwords.txt
banned_words="$script_dir/bannedwords.txt"

if [[ ! -f "$banned_words" ]]; then
  echo "bannedwords.txt not found!"
  exit 1
fi


result=""

# For every word in banned words, do all of this
while IFS= read -r query; do 
  # If the line isn't empty
  if [[ -n "$query" ]]; then
    # In path, find a type (d = dir, f = file), ignore case, matching this regex string
    # Also if didn't find anything don't append a \n
    dir_result=$(find "$path" -type d -iname "*$query*")
    if [[ -n "$dir_result" ]] then
      result+="$dir_result\n"
    fi

    file_result=$(find "$path" -type f -iname "*$query*")
    if [[ -n "$file_result" ]] then
      result+="$file_result\n"
    fi
  fi
done < "$banned_words"

# Search path with bannedwords.txt in mind
# r = recursive, n = print line number, l = stop searching if the FILE has the query, i = ignore case
# f = keywords from file 1 to search for
# Using ripgrep though so r doesn't matter
result+=$(rg -n -i -f "$banned_words" "$path")

# Check if "result" is not empty. This removes the random newlines when no results are found
if [[ -n "$result" ]] then
  echo "$result"
  printf "\n"
fi

(Nix Alias Attempt 1)
# error: getting status of '/nix/store/blahblahblah-source/modules/scripts/CheckNames/bannedwords.txt': No such file or directory
{ pkgs, ... }:

let
  check-names = pkgs.stdenv.mkDerivation {
    name = "check-names";
    src = ./CheckNames.sh; # The script

    # The script requires bannedwords.txt, so let's include it in the output directory
    bannedwords = ./bannedwords.txt;

    installPhase = ''
      mkdir -p $out/bin
      cp $src $out/bin/check-names.sh
      cp $bannedwords $out/bin/bannedwords.txt
      chmod +x $out/bin/check-names.sh
    '';

  };
in
{
  environment.systemPackages = [
    check-names
  ];
}
(Nix Alias Attempt 2)
# error: getting status of '/nix/store/blahblahblah-source/modules/scripts/CheckNames/bannedwords.txt': No such file or directory
{ pkgs, ... }:
let
  banned-words = ./bannedwords.txt;

  # Write the CheckNames.sh script and reference the bannedwords.txt from the store
  check-names = pkgs.writeScriptBin "check-names" ''
    #!/bin/bash
    bannedwordsfile=${banned-words}
    bash ${./CheckNames.sh} "$bannedwordsfile"
  '';  
in {
  environment.systemPackages = [
    check-names
  ];
}
(Nix Alias Attempt 3 and so on)
# error: getting status of '/nix/store/blahblahblah-source/modules/scripts/CheckNames/bannedwords.txt': No such file or directory
{ pkgs, ... }:
let
  bannedWordsFile = pkgs.writeTextFile {
    name = "bannedwords.txt";
    text = builtins.readFile ./bannedwords.txt;
  };

  check-names = { pkgs, ... }:
  pkgs.writeScriptBin "check-names" ''
    #!/bin/bash
    bannedwordsfile=${bannedWordsFile}
    bash ${builtins.readFile ./CheckNames.sh} "$bannedwordsfile"
  '';
in {
  environment.systemPackages = [
    (check-names pkgs)
  ];
}

Why not simply take the words file as a plain old bash script parameter?

I think the writeScriptBin builder might be a bit too specialized for your needs. It’s useful for packaging simple scripts, but not for several files. If you want to avoid making a whole mkDerivation statement, you might be able to get away with something like runCommand and putting the files where you need them to be in $out. See Trivial Builders for more options.

On a side note, you might make your life easier if you allowed the script to find its text file via an environment variable, or leaving a placeholder so you could inject ${placeholder "out"}/path/to/bannedwords.txt path into the script before putting it in $out/bin.

1 Like

Never thought of that!

Though sadly I think that approach isn’t the perfect solution:

  • If you mean typing out all the words that I would like to find/ban, it would be about 10 words typed every time and I then a risk I might forget some
  • If you mean type out the path to the bannedwords.txt file, that would also be a lengthy process since the directory is nested in about 5 layers and I would have to remember the path

However, since I’ve learned what the problem was (Discussed in the next reply) this is probably the best solution for now.

What was the problem?: I didn’t add it to git. Nix really doesn’t like it when you try accessing a file out of git, and since I purposefully gitignored the bannedwords.txt (making sure they don’t get on remote servers) nix wouldn’t find it.

I have thought of a couple solutions but ultimately only two (probably) work:

  1. I could do what @Atemu suggested and just put the banned words as a command-line argument, whether it be a list of words or a path to a list of words. This is my current approach for now.
  2. In the future I could use sops-nix for secrets management, and just put the bannedwords.txt in there as it’s built for handling secrets like this. However this addition would require a lot of refactoring so that’s for another day.
  3. I could create an environment variable with either the path or the actual contents of bannedwords and require the script to look at that variable. However, that would require creating a separate script to put the bannedwords into an environment variable, and it will fail at nix compile time. (Look below for details)
  4. I could copy bannedwords.txt’s contents and store it in a nix variable, then write it into the directory of the bash script. Problem: (Look below for details)

Details: as I understand it now, nix copies the entire repository excluding stuff not covered by git to another place, and then executes all the nix code. This means bannedwords.txt can never be referenced locally by any nix or bash script because bannedwords.txt was gitignored and was never copied to this protected space.

I won’t mark this as “answered” yet because this is all still theoretical and there’s probably still some discussion to be had.