Rust App in Nix Docker container unable to spawn process?

Consider the following Rust app

fn main() -> Result<(), Box<dyn Error>> {                                                                                                                                                                                      
    process::Command::new("echo").arg("hello")                                                                                                                   
          .status()?;    
}

and container

      docker = pkgs.dockerTools.buildImage {                                                                                                                                                                                      
        name = "my-app";                                                                                                                                                                                                     
        tag = "latest";                                                                                                                                                                                                           
                                                                                                                                                                                                                                  
        # everything in this is *copied* to the root of the image                                                                                                                                                                 
        contents = [                                                                                                                                                                                                              
          self.app                                                                  
          pkgs.glibc                                                                                                                                                                                                              
          pkgs.coreutils                                                                                                                                                                                                          
          pkgs.runtimeShellPackage                                                                                                                                                                                                
        ];                                                                                                                                                                                                                        
                                                                                                                                                                                                                                  
        # run unprivileged with the current directory as the root of the image                                                                                                                                                    
        extraCommands = ''                                                                                                                                                                                                        
          #!${pkgs.runtimeShell}                                                                                                                                                                                                  
          mkdir -p /tmp                                                                                                                                                                                       
        '';                                                                                                                                                                                                                       
                                                                                                                                                                                                                                  
        # Docker settings                                                                                                                                                                                                         
        config = {                                                                                                                                                                                                                
          Cmd = [ "app" ];                                                                                                                                                                                                
          Volumes = {                                                                                                                                                                                                             
            "/tmp" = { };                                                                                                                                                                                                         
          };                                                                                                                                                                                                                      
        };                                                                                                                                                                                                                        
      }; 

where app is the derivation of the rust app.

nix build -j auto ".#docker" && docker image load -i result;

so far so good. But then!

docker run -it my-app

Fails with

Error: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" }
error: Operation not permitted

Whaaat? docker run -it dots-docker echo hello works just fine! binary works fine out of the container. Executable bit is definitely set.

This could be a docker problem (or rust), but I have a sneaking suspicion it’s a library mapping issue? Something is obviously wrong here, and just wanted to probe for ideas. Let me know if this is enough for replication. Is there something obvious going on? Happy to hear your thoughts.

Just this week I spent a day debugging this error in Jenkins. In my case when I ran strace I saw an EPERM error from clone3. If you can get a trace and you find that error, it may be a Docker bug. See the links in Docker in Docker cannot install bundler (ruby gem), on alpine 3.14+ · Issue #43311 · moby/moby · GitHub and the 20.10.10 release notes. The solution was to upgrade the Docker version.

Of course with such a generic error it could be anything, but either way getting a trace might be a good place to start.

1 Like

A friend of mine had some issues with clone3 run by a rust executable inside docker, we solved by adding --security-opt seccomp=unconfined to the docker run invocation.

See Docker.io with the clone3 fix : Pascal Roeleven

1 Like

Wow. Glad I asked opposed to trying to rediscover this bug. clone3 indeed seems to be the problem. Provided the solution to azazel75 since austin is already a great contributor!!

Thanks!