Version mismatch with pandoc-types and ob run/ghci

Help
1

I have a weird version mismatch, when trying to run ob run, which is a tool of the obelisk framework and it invokes ghci.

/home/ruben/code/my-palantype/learn-palantype/dist-newstyle/tmp/src-125670/pandoc-types-1.22.2/src/Text/Pandoc/Definition.hs:100:1: error:
    Could not load module ‘Paths_pandoc_types’
    It is a member of the hidden package ‘pandoc-types-1.23’.
    You can run ‘:set -package pandoc-types’ to expose it.
    (Note: this unloads all the modules in the current scope.)
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
    |
100 | import Paths_pandoc_types (version)
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

For some reason, I have a dependency to pandoc-types-1.22 that than believes that it depends on some pandoc-types-1.23 for the module Paths_pandoc-types. It doesnt make sense. I should only have pandoc-types-1.23 in the first place.

I posted an issue there but to me it seems unlikely that this is caused by a bug in obelisk.

I’d rather like to know how a situation comes about in which two haskell packages appear in different versions in the same derivation. And maybe this way I know where to look for the source of this error.

This is my default.nix (the configuration of my obelisk project). Among other things, in this file I specifically select the version pandoc-types-1.23.

{ system ? builtins.currentSystem
, pkgs ? import <nixos-unstable> {}
}:
let
  obelisk = (import ./.obelisk/impl {
    inherit system;
    iosSdkVersion = "13.2";
    terms.security.acme.acceptTerms = true;
    useGHC810 = true;
  });

in
  with pkgs.haskell.lib;
  obelisk.project ./. ({ ... }: {
    android.applicationId = "systems.obsidian.obelisk.examples.minimal";
    android.displayName = "Palantype";
    ios.bundleIdentifier = "systems.obsidian.obelisk.examples.minimal";
    ios.bundleName = "Palantype";

    staticFiles = import ./static.nix { inherit pkgs; };

    shellToolOverrides = self: super: {
      inherit (pkgs.haskellPackages) cabal-plan cabal-install;
    };

    overrides = self: super: {

      mkDerivation = args: super.mkDerivation (args // {
          doCheck = false;
          doHaddock = false;
          enableLibraryProfiling = false;
        });

      # for persistent
      lift-type = self.callHackage "lift-type" "0.1.0.1" {};
      # esqueleto's test suite introduces dependencies to persistent-... packages
      esqueleto = self.callHackage "esqueleto" "3.5.3.0" {};
      persistent = self.callHackage "persistent" "2.13.2.1" {};
      # the version 2.13.0.2 still got a bug with mariadb
      #persistent-mysql = self.callHackage "persistent-mysql" "2.13.0.2" {};
      persistent-mysql = self.callHackageDirect {
        pkg = "persistent-mysql";
        ver = "2.13.0.3";
        sha256 = "0mnrgq05nk7ghfavfr2rhcz2yc0lciw88idri0ljsk2nymsnrbb8";
      } {};

      gerippe = self.callCabal2nix "gerippe" (pkgs.fetchFromGitHub {
        owner = "rubenmoor";
        repo = "gerippe";
        rev = "041f32de5094589ebc7dfe2bfd6e8078470ef2b6";
        sha256 = "1mfxxn0q09wa81bdw4ngisklaaib4lz1ibxayrpk7g7l44hy92sm";
      }) {};
      #gerippe = self.callCabal2nix "gerippe" ../../gerippe {};

      gridtables = self.callHackageDirect {
        pkg = "gridtables";
        ver = "0.1.0.0";
        sha256 = "zMQW1jbntZ1pxWKT+D+tt/LKy7SkADFN3o+Wn3VdD88=";
      } {};

      jira-wiki-markup = self.callHackageDirect {
        pkg = "jira-wiki-markup";
        ver = "1.5.0";
        sha256 = "a0/b2AfcA7mMmfrcbEEh5pNUq/OjJw4H7uLTpkl+Y1c=";
      } {};
      pandoc-types = self.callCabal2nix "pandoc-types" ../pandoc-types {};
      #pandoc-types = self.callHackageDirect {
      #  pkg = "pandoc-types";
      #  ver = "1.23";
      #  sha256 = "BfwTC6irH6/dYIuqd+rMqLNzH8DP1g/7mFJhZZ9O9Fo=";
      #} {};

      commonmark-pandoc = self.callHackageDirect {
        pkg = "commonmark-pandoc";
        ver = "0.2.1.3";
        sha256 = "0JXjvqZVFa8RR64Wi9NZ97tQ9S900qRaE6/4adQaQaU=";
      } {};
      commonmark = self.callHackage "commonmark" "0.2.2" {};
      commonmark-extensions = self.callHackageDirect {
        pkg = "commonmark-extensions";
        ver = "0.2.3.3";
        sha256 = "J0sdUT4X0yjJ4iAu/ne7EA4IzFAoyBWf59hx5FyOsH4=";
      } {};
      unicode-data = self.callHackageDirect {
        pkg = "unicode-data";
        ver = "0.3.1";
        sha256 = "p3uoLleamcIYEqe9c+EFRO3mPMZGgLf9XQ3/qwdmiF8=";
      } {};

      texmath = self.callHackageDirect {
        pkg = "texmath";
        ver = "0.12.6";
        sha256 = "r5R3TfWOYfgMH4JDXMX14JPykGjKXuJn6IovJ4rSnTo=";
      } {};
      citeproc = self.callHackageDirect {
        pkg = "citeproc";
        ver = "0.8.1";
        sha256 = "XhaAQo0kYCZzenyUftgiCqNs/5esca2udcfeSpBOTjI=";
      } {};

      doclayout = self.callHackage "doclayout" "0.4" {};
      emojis = self.callHackage "emojis" "0.1.2" {};
      doctemplates = self.callHackageDirect {
        pkg = "doctemplates";
        ver = "0.11";
        sha256 = "8xT3CJzWjhLEHTBcbl+BCF0lc5fVQUQ0IRdLMNmthmc=";
      } {};
      ipynb = self.callHackage "ipynb" "0.2" {};
      mime-types = self.callHackageDirect {
        pkg = "mime-types";
        ver = "0.1.1.0";
        sha256 = "HnBsPBdONbaw42/JnpONXAIwsEwc/fMyUXcm4cJdWUQ=";
      } {};

      pandoc = self.callCabal2nix "pandoc" (pkgs.fetchFromGitHub {
        owner = "rubenmoor";
        repo = "pandoc";
        rev = "8fa46a2e5701db1ed579cc65378a927746ca8586";
        sha256 = "0vl1ikgg80rkwdz6chj2s2h5xdkylvi4ymcziq3v0s62pnqf8bk3";
      }) {};

      skylighting-core = self.callHackage "skylighting-core" "0.9" {};
      skylighting = self.callHackage "skylighting" "0.9" {};

      #reflex-dom-pandoc = self.callHackage "reflex-dom-pandoc" "1.0.0.0" {};
      # waiting for my pull request to make it into upstream
      reflex-dom-pandoc = self.callCabal2nix "reflex-dom-pandoc" (pkgs.fetchFromGitHub {
        owner = "rubenmoor";
        repo = "reflex-dom-pandoc";
        rev = "7878b040d15ba9d327a56e673231c6467d7c5973";
        sha256 = "111x4qs958jq38vrm56m4jl4jrww3jjw2kyd2l2clwmxp7wipkz0";
      }) {};

      #servant-reflex = self.callCabal2nix "servant-reflex" ../servant-reflex {};
      servant-reflex = self.callCabal2nix "servant-reflex" (pkgs.fetchFromGitHub {
        owner = "rubenmoor";
        repo = "servant-reflex";
        rev = "00b101c46fb3ecb3d35fa55080b4b32455cee66e";
        sha256 = "1gghpfgsw8p2v6msz17sr5nn30pkg2c59pz7gvazqrv80v8102bb";
      }) {};

      #servant-snap = self.callCabal2nix "servant-snap" ../servant-snap {};
      servant-snap = self.callCabal2nix "servant-snap" (pkgs.fetchFromGitHub {
        owner = "rubenmoor";
        repo = "servant-snap";
        rev = "657af13efa002b4f7c24bfc20d63d59e85a1086f";
        sha256 = "0p1h1a1rnrg5c63cpir8i26w93af6hqwxgqc6w2h94wjz2fbxp4c";
      }) {};

      my-palantype = self.callCabal2nix "my-palantype" ../my-palantype { };
      #my-palantype = self.callCabal2nix "my-palantype" (pkgs.fetchFromGitHub {
      #  owner = "rubenmoor";
      #  repo = "my-palantype";
      #  rev = "c258a5f678011dde1ff24ead2b5574c61d6398cb";
      #  sha256 = "0hqd5vbc8wamm4npvsvg06vghv7vcin6pmx15wd9lysi3q7zf2s4";
      #}) {};

      bytestring-trie = self.callHackage "bytestring-trie" "0.2.7" {};
    };
  })

Something about this is specific to ob run/ghci. nix-build -A exe works fine and so does nix-shell -A shells.ghc.

2

I don’t know how many people on here know enough of the details of obelisk to be able to help you out. If you don’t get any answers, I’d recommend posting on obelisk’s issue tracker.

Also, you’ll likely get more help if you post your full source-code (ideally a git repo), including what <nixos-unstable> is pointing to, etc. That way other people can try to reproduce your problem.

2 Likes
3

Have you exited your shell and run ob shell again since making dependency/Nix configuration changes? Sometimes a mismatch between the environment created by ob shell and updated configuration files can result in strange issues.

I do not see any issues with the dependencies. I cannot check the pandoc-types package itself, however, because it is specific to your environment:

pandoc-types = self.callCabal2nix "pandoc-types" ../pandoc-types {};

Given that nix-build -A exe works, I assume that ../pandoc-types is a repo with the appropriate version checked out.

1 Like
4

issue on obelisk: pandoc-types does not built with `ob run`, weird version mismatch · Issue #1022 · obsidiansystems/obelisk · GitHub

It’s tricky. Indeed some knowledge of obelisk’s inner workings seems required. At the same time it isn’t clear at all whether this is an obelisk issue.

I set up the repo to reproduce the bug, without any dependencies to local files:

$ git clone git@github.com:rubenmoor/learn-palantype.git
$ cd learn-palantype
$ git checkout pandoc-type-version-mismatch # currently HEAD
$ ob run # ob run -v doesn't produce more helpful output

/home/ruben/code/my-palantype/learn-palantype/dist-newstyle/tmp/src-125670/pandoc-types-1.22.2/src/Text/Pandoc/Definition.hs:100:1: error:
    Could not load module ‘Paths_pandoc_types’
    it is a hidden module in the package ‘pandoc-types-1.23’
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
    |
100 | import Paths_pandoc_types (version)
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Right at the top of the default.nix file I do import <nixos-unstable> {}. This points to the current unstable channel: nixos-unstable release nixos-23.05pre456790.988cc958c57. This is, in principle, a possible error source.

To verify that nix-shell and nix-build work:

nix-build -A exe --no-out-link, nix-shell -A shells.ghc, or ob shell. All of which work fine.

2 Likes
5

Yes. The problem persists

1 Like
6

Thank you for setting up the repo for testing! Please note that nixos-unstable is still a reference to your environment settings, though. How is your nixos-unstable configured?

7

I saw your update to the above comment, specifying the nixos-unstable reference. Checking the hackage-packages.nix, pandoc-types is at version 1.22.2.1. Note that the configured versions of other packages in the Pandoc ecosystem likely require this version. For example, package pandoc is at version 2.19.2, and that version has constraint pandoc-types (>=1.22.2 && <1.23).

Perhaps that is the source of the version issue. One thing to try is to make sure that you are explicitly configuring compatible versions of all of the transitive dependencies of your project that require pandoc-types. Perhaps a transitive dependency is not configured in your default.nix and is therefore using a package from nixos-unstable that is not compatible with the newer version of pandoc-types.

1 Like
8

so even though I have this in my default.nix

pandoc-types = pandoc-types_1_23

there is still some dependency to pandoc-types version 1.22? There aren’t a lot of packages in my project that require pandoc-types. I checked and found

  • pandoc
  • pandoc-reflex-dom

Is there more that I can do to

  • make sure that they only see the pandoc version that I explicitly chose?
  • make the error more explicit (currently, there seems to be an implicit dependency)?
1 Like
9

Moving to pandoc-types 1.22 is most probably a viable workaround. I would like to understand how the mismatch comes about (i.e. explicitly selecting one version and still the older version is still around, messing stuff up)

10

I was able to test this evening, and I found an issue!

File static-css/release.nix sets the default pkgs to <nixpkgs>:

{ pkgs ? import <nixpkgs> {} }:

It is imported from static.nix without overriding that default:

staticCss = import ./static-css/release.nix {};

This means that some packages come from <nixos-unstable> and some packages come from <nixpkgs>. You can resolve this issue using inherit:

[nix-shell:/tmp/learn-palantype]$ git diff static.nix
diff --git a/static.nix b/static.nix
index 38bb9c4..fcac783 100644
--- a/static.nix
+++ b/static.nix
@@ -1,6 +1,6 @@
 { pkgs ? import <nixpkgs> {} }:
 let
-  staticCss = import ./static-css/release.nix {};
+  staticCss = import ./static-css/release.nix { inherit pkgs; };
 in pkgs.stdenv.mkDerivation {
   name = "static";
   src = ./static-files;

With this issue resolved, I was able to run both ob run and ob ghci without errors. I confirmed the version of pandoc-types in the REPL:

[nix-shell:/tmp/learn-palantype]$ ob repl
...
*Backend Obelisk.Run Frontend Backend> :show packages
...
  -package-id pandoc-types-1.23-BhrOhSU8fTJKOJG9dFtPTj
...

Does this fix work for you as well?

11

I pinned the nixpkgs to the master of the github repo NixOS/nixpkgs, instead of just using my locally configured channel “nixos-unstable” for actual reproducibility in the default.nix:

 pkgs = import (builtins.fetchGit {
    name = "nixos-unstable-2023-02-26";
    url = "https://github.com/nixos/nixpkgs/";
    ref = "refs/heads/nixos-unstable";
    rev = "7f5639fa3b68054ca0b062866dc62b22c3f11505";
  }) {};

Your fix of statix.nix is interesting, but didn’t resolve the issue for me. ob run and ob repl still give me the same mixup:

/home/ruben/code/my-palantype/learn-palantype/dist-newstyle/tmp/src-125670/pandoc-types-1.22.2/src/Text/Pandoc/Definition.hs:100:1: error:
    Could not load module ‘Paths_pandoc_types’
    it is a hidden module in the package ‘pandoc-types-1.23’
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.

When I check for the version of pandoc-types within the shell, using cabal-plan, I get

$ nix-shell -A shells.ghc
$ cabal-plan info | grep pandoc-types     
 │   │   ├─ pandoc-types-1.23
 │   │   ├─ pandoc-types-1.23 ┄┄
 │   │   │   ├─ pandoc-types-1.23 ┄┄
 │   │   │   ├─ pandoc-types-1.23 ┄┄
 │   │   │   ├─ pandoc-types-1.23 ┄┄
 │   │   ├─ pandoc-types-1.23 ┄┄
 │   │   │   ├─ pandoc-types-1.23 ┄┄
 │   ├─ pandoc-types-1.23 ┄┄
UnitId "pandoc-types-1.23-BhrOhSU8fTJKOJG9dFtPTj"
  pandoc-types-1.23
  pandoc-types-1.23
  pandoc-types-1.23
12

currently, I assume that haskellPackages.pandoc in the nix packages is stuck at version 2.19.2 for a reason. Probably it’s not possible to get it to work properly just via callCabal2nix, but it rather relies on some more work.

13

That it works fine on my system yet still fails on yours may indicate that it is a difference in environment. Pinning nixpkgs is a good idea, IMHO. Perhaps you can try ensuring that pkgs is consistent across all of the .nix files in the project, as one of the static .nix files may still be referencing your environment.

1 Like
14

The fact that it worked on your system is the solution.

I ran

$ nix-shell -A shells.ghc
$ cabal clean

and now ob run works fine.

ob run depends on former runs of cabal build ...

Thanks for trying to reproduce!! That solved the problem.

1 Like