Go weirdness with nix-shell

What is this nix-shell weirdness…

I have go in environment.systemPackages but unless I run it through nix-shell -p go my code doesn’t run :confused:

Have a look at this.
In short:

  • initialize Go module
  • download dependencies
  • run. Fails.
  • enter nix-shell
  • run the exact same code - Works
    (go version is the same inside and outside nix-shell)
[marius@nixos:~/dev/go-projects/kb]$ ls
main.go

[marius@nixos:~/dev/go-projects/kb]$ go mod init myprogram
go: creating new go.mod: module myprogram
go: to add module requirements and sums:
	go mod tidy

[marius@nixos:~/dev/go-projects/kb]$ go mod tidy
go: finding module for package github.com/micmonay/keybd_event
go: downloading github.com/micmonay/keybd_event v1.1.2
go: found github.com/micmonay/keybd_event in github.com/micmonay/keybd_event v1.1.2

[marius@nixos:~/dev/go-projects/kb]$ go run main.go 
# github.com/micmonay/keybd_event
../../../go/pkg/mod/github.com/micmonay/keybd_event@v1.1.2/keybd_event.go:20:9: undefined: initKeyBD

[marius@nixos:~/dev/go-projects/kb]$ nix-shell -p go

[nix-shell:~/dev/go-projects/kb]$ go run main.go 

[nix-shell:~/dev/go-projects/kb]$ 

What is nix-shell doing that I am missing?

Here is the program if anyone wants to try. It should be run with sudo to give output, but I am omitting it here to keep the complexity of the post to a minimum.

package main

import (
	"time"
	"github.com/micmonay/keybd_event"
)

func main() {
	kb, _ := keybd_event.NewKeyBonding()
	time.Sleep(2 * time.Second)
	
	// Select keys to be pressed
	kb.SetKeys(keybd_event.VK_A, keybd_event.VK_B) 

	// Set shift to be pressed
	kb.HasSHIFT(true) 

	// Press the selected keys
	kb.Launching() 
	
	// Here, the program will generate "AB" as if they were pressed on the keyboard.
}

1 Like

What’s the output of go env (for both)?

That’s not where dev packages (libraries or dev binaries) are meant to go; you really are meant to use dev shells (nix-shell or nix develop).

Outside nix-shell

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/marius/.cache/go-build'
GOENV='/home/marius/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/marius/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/marius/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/nix/store/gdp1f96y3rhrpncllbngi1chmpc9k0gd-go-1.22.6/share/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/nix/store/gdp1f96y3rhrpncllbngi1chmpc9k0gd-go-1.22.6/share/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.6'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='0'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build1035197357=/tmp/go-build -gno-record-gcc-switches'

Inside nix-shell -p go:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/marius/.cache/go-build'
GOENV='/home/marius/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/marius/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/marius/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/nix/store/gdp1f96y3rhrpncllbngi1chmpc9k0gd-go-1.22.6/share/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/nix/store/gdp1f96y3rhrpncllbngi1chmpc9k0gd-go-1.22.6/share/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.6'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build822061630=/tmp/go-build -gno-record-gcc-switches'

And the diff

[nix-shell:~/Documents]$ diff 1 2
31c31
< CGO_ENABLED='0'
---
> CGO_ENABLED='1'
40c40
< GOGCCFLAGS='-fPIC -m64 -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build773552264=/tmp/go-build -gno-record-gcc-switches'
---
> GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3235042170=/tmp/go-build -gno-record-gcc-switches'

Hm interesting! Am I really not supposed to install any dev environment as a system package? How come? Any link to more info?

Due to the exact sort of issue you report here.
Also, why even use nix if you want to install dev things globally?

Well, why install anything if you can just nix-shell them?

To me, I’d like it installed because it is something I will be using regularly. I’m not a developer so I don’t need several different environments with different versions and stuff.

I don’t see why this should be controversial or wrong

1 Like

The whole point of nix is correct deployments; each package sees its own set of packages and nothing else. If putting libraries in a global namespace just worked that would defeat the whole purpose of nix.
Again, why did you switch to using Nix and NixOS if that’s not the behaviour you wanted?

A non-dev wouldn’t be using go in the first place :wink:

Maybe we shouldn’t talk to people like this?

There’s nothing wrong with expecting thing to work the same way it does in every other Linux distribution.

4 Likes

In a mkShell, your shell also contains the C toolchain which is needed for Cgo to work. The library you’re using requires Cgo (as evident by import "C" in the keybd_linux.go file). The Go toolchain on its own does not come with the C toolchain.

I’m not sure how to get around this, as you clearly need more than just the Go toolchain in the global environment, but I recommend only to assume that the global Go toolchain is enough when the project has no Cgo, otherwise you would need a Nix shell environment, since you probably also want extra C libraries anyway.

6 Likes

Can confirm, tried pkgs.mkShellNoCC and go build fails with the same issue as globally installed go due to c compiler being unavailable.

{ pkgs ? import <nixpkgs> { } }:
pkgs.mkShellNoCC {
   packages = [ pkgs.go ];
   env.CGO_ENABLED = "1"; # fails with a clearer error message
}
$ go build .
# runtime/cgo
cgo: C compiler "gcc" not found: exec: "gcc": executable file not found in $PATH

So as waffle and diamond said, it is still best to use project level shell.nix or flake.nix with pkgs.mkShell.

If you are worried about having multiple go binaries installed and taking up storage space you may use the <nixpkgs> import (called diamond import) to keep your go binary same as the system level binary.
But I still recommend pinning nixpkgs and thus the go version using flakes or npins+shell.nix per project.

1 Like

I’m asking what their motivation is. NixOS is not just every other Linux distribution, that is its “selling point”. It’s a massive disservice to pretend to beginners that NixOS is the same as any other distro with respect to how development and configuration is done. (Of course the contents of config files can be the same or similar, but how that config is written is also NixOS specific in many cases.)

I genuinely don’t understand why people expect to use dev environments globally when switching to NixOS, when the selling point is “correct deployments”.

Because the new users are still used to the old way of doing things from prev os and it takes time to figure out the best practices.

There’s nothing wrong with what you’ve said in this thread so far, but the tone comes across a bit stern, that’s all.

@marius you may mark diamondburned’s explaination as the answer.

4 Likes

Fair enough, perhaps I’m lost in the sauce. No harm meant to anyone, OP included.

3 Likes

Is mkShell some function that is run when I invoke nix-shell? And it adds “build-essentials” automatically?

This was the hint I needed - spot on!

After adding gcc to environment.systemPackages everything works as expected.

Now, I understand this is not how I’m supposed to be doing it so I’ll be doing some more digging.

This goes right over my head for now :sweat_smile: Any advice on where to go next to read up on this stuff?

And thanks for your replies, all!

nix.dev has a tutorial on using pkgs.mkShell with a shell.nix file: Declarative shell environments with shell.nix — nix.dev documentation

2 Likes

I believe it’s either mkShell or mkDerivation that nix-shell invokes, but both of them belong under stdenv which by default comes with the C toolchain as part of how it works. You can understand it as build-essentials, but there will be some differences as to what is and isn’t included.

This article works: Development environment with nix-shell - NixOS Wiki

If you search up “nix shell”, you’ll get a bunch of resources for things that vaguely do the same job but are completely different tools, such as:

  • The nix shell command
  • The nix develop command
  • The nix-shell command
  • The devShells part of a flake.nix
  • Nix flakes themselves
  • The shell.nix file

For beginners, I recommend either picking the flake.nix route or the shell.nix route, but you have to pay attention that they’re different things that will show up for the same search query.

2 Likes