Nixops and Tomcat

Help
1

I am learning nix and nixops. I am on a Mac, and am using VirtualBox to run the nixos machines. I am using a Docker image I found here as my remote builder.

I want to use Tomcat. I read the module for Tomcat and think that all I need to do is write something like this

services.tomcat.enable = true;
services.tomcat.webapps = [ ./hello.war ];
networking.firewall.allowedTCPPorts = [ 8080 ];

Tomcat gets started up, and when I try to talk to it on port 8080 it responds and says that there is no “hello” app. I can look in /var/tomcat (which is where the tomcat module says everything should be) and there is in fact no hello.war file or hello app. I can read all the log files, which seem normal assuming there is no hello.war file. I thought all I had to do to get it loaded with the appropriate war files was to list them as above, but apparently there is something else I need to do that I couldn’t figure out by reading the module. I haven’t found any examples of using Tomcat for nix, so I am starting to wonder whether the Tomcat module actually works. Does anybody have an example of nixops file using Tomcat that they would share with me?

2

The module should work as our tomcat test still passes on hydra: https://hydra.nixos.org/eval/1490490?filter=tomcat&compare=1490483&full=

cc @danbst @pvgoran who recently worked on this pull request nixos/tomcat: add purifyOnStart option by pvgoran · Pull Request #44303 · NixOS/nixpkgs · GitHub

1 Like
3

The test doesn’t seem to use services.tomcat.webapps. It loads war files some other way. Using curl. I didn’t understand how the test worked, which is not surprising, and not really a problem that needs fixing. However, I would like to see an example that works and that uses services.tomcat.webapps.

4

I do use Tomcat with NixOps, but my setup is very different:

  • I work with expanded webapps, rather than with .war files.
  • I specify the webapps as plain strings (rather than Nix paths), these strings specify paths in the target machine’s filesystem, and Web applications are deployed there imperatively.

Please paste the result of ls -l /var/tomcat/webapps (on the target machine, of course).

5

The result is

total 0

It is an empty directory.

6

There are a few problem with WAR deployment how you do it.

First is that there is default value for tomcat.webapps

And by specifying tomcat.webapps = [ ./hello.war ]; you overwrite it. So your localhost:8080/ stops responding (because ROOT webapp was part of default tomcat webapps)

Next, war-files dropped to webapps are exposed under their .war name. But ./hello.war will have /nix/store/blaballonghash-hello.war name, so will be exposed under http://localhost:8080/blaballonghash-hello path. This is not what you want, you wanted http://localhost:8080/hello. The solution here is to wrap hello.war into a directory

let 
  hello-in-dir = pkgs.runCommand "hello.war" {} ''
    mkdir -p $out/webapps/
    ln -s ${./hello.war} $out/webapps/hello.war
  '';
in {
  ...
  services.tomcat.webapps = [ hello-in-dir ];
}

This behavior is defined in

After that Tomcat automatically unpacks WAR file into directory with same name and you can access it on HTTP.

But you say there is no hello.war file. Can you du -a /var/tomcat, journalctl -u tomcat and cat /var/tomcat/logs/catalina.out?

1 Like
7

Yes, I too recommend imperative deployment. Mostly because it fits CI better. Also tomcat can do rolling versioned deploy without service restart, while current Tomcat Nixos module does service restart on each change. So I always specify systemd.services.tomcat.restartIfChanged = mkOverride 200 false; and restart tomcat imperatively.

8

I am using nix because I want it all to be declarative. In my use case, I am happy to restart Tomcat. I’m not developing a normal web site, and in fact I will probably switch out Tomcat at some point for a lighter weight solution. But it is the server I know best so I wanted to learn nix with it.

Here is du -a /var/tomcat

4 /var/tomcat/shared/lib
8 /var/tomcat/shared
4 /var/tomcat/work
4 /var/tomcat/lib/websocket-api.jar
4 /var/tomcat/lib/tomcat-i18n-fr.jar
4 /var/tomcat/lib/tomcat-util-scan.jar
4 /var/tomcat/lib/servlet-api.jar
4 /var/tomcat/lib/jsp-api.jar
4 /var/tomcat/lib/annotations-api.jar
4 /var/tomcat/lib/tomcat-i18n-es.jar
4 /var/tomcat/lib/jasper-el.jar
4 /var/tomcat/lib/ecj-4.6.3.jar
4 /var/tomcat/lib/tomcat-dbcp.jar
4 /var/tomcat/lib/tomcat-i18n-ja.jar
4 /var/tomcat/lib/catalina-ant.jar
4 /var/tomcat/lib/catalina-storeconfig.jar
4 /var/tomcat/lib/tomcat-jdbc.jar
4 /var/tomcat/lib/jaspic-api.jar
4 /var/tomcat/lib/tomcat-jni.jar
4 /var/tomcat/lib/tomcat-util.jar
4 /var/tomcat/lib/el-api.jar
4 /var/tomcat/lib/catalina-ha.jar
4 /var/tomcat/lib/tomcat-coyote.jar
4 /var/tomcat/lib/catalina.jar
4 /var/tomcat/lib/jasper.jar
4 /var/tomcat/lib/catalina-tribes.jar
4 /var/tomcat/lib/tomcat-websocket.jar
4 /var/tomcat/lib/tomcat-api.jar
104 /var/tomcat/lib
0 /var/tomcat/logs/host-manager.2018-11-17.log
0 /var/tomcat/logs/localhost.2018-11-19.log
16 /var/tomcat/logs/catalina.2018-11-17.log
36 /var/tomcat/logs/catalina.out
16 /var/tomcat/logs/catalina.2018-11-19.log
0 /var/tomcat/logs/manager.2018-11-17.log
4 /var/tomcat/logs/localhost_access_log.2018-11-17.txt
0 /var/tomcat/logs/manager.2018-11-19.log
0 /var/tomcat/logs/localhost.2018-11-17.log
0 /var/tomcat/logs/host-manager.2018-11-19.log
4 /var/tomcat/logs/localhost_access_log.2018-11-19.txt
80 /var/tomcat/logs
4 /var/tomcat/bin
4 /var/tomcat/virtualhosts
4 /var/tomcat/webapps
4 /var/tomcat/temp
4 /var/tomcat/conf/catalina.policy
4 /var/tomcat/conf/logging.properties
8 /var/tomcat/conf/catalina.properties
4 /var/tomcat/conf/context.xml
4 /var/tomcat/conf/Catalina/localhost
8 /var/tomcat/conf/Catalina
8 /var/tomcat/conf/server.xml
4 /var/tomcat/conf/tomcat-users.xml
4 /var/tomcat/conf/tomcat-users.xsd
4 /var/tomcat/conf/web.xml
4 /var/tomcat/conf/jaspic-providers.xml
4 /var/tomcat/conf/jaspic-providers.xsd
60 /var/tomcat/conf
276 /var/tomcat

journalctl -u tomcat has a bunch of starting and stopping tomcat, nothing else.

The log “catalina.out” has no error messages. I’ve started the server a bunch of times and it always looks like this.

19-Nov-2018 17:41:04.990 INFO [main] org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler [“http-nio-8080”]
19-Nov-2018 17:41:05.063 INFO [main] org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler [“ajp-nio-8009”]
19-Nov-2018 17:41:05.118 INFO [main] org.apache.catalina.core.StandardService.stopInternal Stopping service [Catalina]
19-Nov-2018 17:41:05.123 INFO [main] org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler [“http-nio-8080”]
19-Nov-2018 17:41:05.136 INFO [main] org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler [“ajp-nio-8009”]
19-Nov-2018 17:41:05.163 INFO [main] org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler [“http-nio-8080”]
19-Nov-2018 17:41:05.167 INFO [main] org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler [“ajp-nio-8009”]
19-Nov-2018 17:41:15.726 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version: Apache Tomcat/8.5.23
19-Nov-2018 17:41:15.736 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built: Sep 28 2017 10:30:11 UTC
19-Nov-2018 17:41:15.737 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server number: 8.5.23.0
19-Nov-2018 17:41:15.738 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name: Linux
19-Nov-2018 17:41:15.738 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version: 4.14.74
19-Nov-2018 17:41:15.739 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture: amd64
19-Nov-2018 17:41:15.739 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home: /nix/store/70v8s869f6hx89x8jx5wbbp1xi8nak7r-openjdk-8u181b13/lib/openjdk/jre
19-Nov-2018 17:41:15.740 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version: 1.8.0_181-13
19-Nov-2018 17:41:15.742 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor: Oracle Corporation
19-Nov-2018 17:41:15.742 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE: /var/tomcat
19-Nov-2018 17:41:15.743 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME: /nix/store/xqksjaccrz7l8qwbd34qj5p1qdlkndx0-apache-tomcat-8.5.23
19-Nov-2018 17:41:15.744 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/var/tomcat/conf/logging.properties
19-Nov-2018 17:41:15.745 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
19-Nov-2018 17:41:15.746 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djdk.tls.ephemeralDHKeySize=2048
19-Nov-2018 17:41:15.749 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
19-Nov-2018 17:41:15.749 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/var/tomcat
19-Nov-2018 17:41:15.750 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/nix/store/xqksjaccrz7l8qwbd34qj5p1qdlkndx0-apache-tomcat-8.5.23
19-Nov-2018 17:41:15.751 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/var/tomcat/temp
19-Nov-2018 17:41:15.752 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: [/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib]
19-Nov-2018 17:41:16.021 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler [“http-nio-8080”]
19-Nov-2018 17:41:16.054 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
19-Nov-2018 17:41:16.064 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler [“ajp-nio-8009”]
19-Nov-2018 17:41:16.068 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
19-Nov-2018 17:41:16.078 INFO [main] org.apache.catalina.startup.Catalina.load Initialization processed in 1370 ms
19-Nov-2018 17:41:16.144 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina]
19-Nov-2018 17:41:16.144 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/8.5.23
19-Nov-2018 17:41:16.191 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler [“http-nio-8080”]
19-Nov-2018 17:41:16.222 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler [“ajp-nio-8009”]
19-Nov-2018 17:41:16.245 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 167 ms

I’ve been reading tomcat.nix. I am a nix beginner, but am comfortable with functional programming. It looks to me like if the path is a directory, it looks in a subdirectory called webapps and adds a symbolical link to each file from /var/tomcat/webapps. So, I tried putting hello.war in a directory called webapps in a directory called apps, and naming apps in the list of webapps. Still nothing.

9

You are right about what I put in overwriting the default “webapps” directory. When I commented out the “services.tomcat.webapps” line, tomcat started displaying all the usual files.

10

In fact, the tomcat modules uses the webapps option in a way that will not automatically transfer the files/directories that are mentioned in it to the target host. Specifically, the expression toString cfg.webapps seems to expand to original paths of the mentioned files, rather than to the paths copied to Nix store.

As a work-around, it should be possible to force copying of the .war file to store in your configuration file rather than in the tomcat module, by using the following construct:

services.tomcat.webapps = [ "${./hello.war}" ];

However, after this you will face the problem mentioned by @danbst: the name of the .war file will include the hash, rather than being just hello.war. So you will still need to wrap the file into a directory to preserve its name:

services.tomcat.webapps = [ "${./directory-with-webapps-and-then-hello-war}" ];

Or use pkgs.runCommand as suggested by @danbst, if you don’t want to keep your hello.war file inside the expected directory structure.

11

That worked! Now /var/tomcat/webapps has hello.war and the unpacked hello directory. I am happy to wrap the war file in a directory structure to make it work. I understand why that is necessary.

However, I don’t understand why “${./apps}” works. This should be a string, not a path. Doesn’t this expression get evaluated on my Mac rather than the nixos machine? If the script uses a path, I can imagine that nixops copies the entire directory structure under a path when it passes the path from the Mac to the nixos machine. But this is string, so it will just get passed as a string. This doesn’t make sense to me.

1 Like
12

As far as I understand, the entire Nix configuration is evaluated on the host machine where NixOps is run (a Mac in your case); this creates, among other things, the Tomcat initialization script file (in the local Nix store) that is sent to the target machine. This script file (or any built file, for that matter) can contain references to other files/directories in the local Nix store; Nix knows it should then send these referenced objects to the target machine as well.

So, if you specify ./hello.war, evaluation of the configuration will produce a Tomcat initialization script file that contains the string /path/to/your/hello.war. This is not a Nix store path, so it won’t be caught by Nix. However, the "${./hello.war}" form copies the file (or directory) to the Nix store (locally!) and results in the store path; this store path gets included in Tomcat initialization script file, Nix notices this and copies the referenced store path to the target machine where it is accessed by the Tomcat initialization script.

1 Like
13

So yes, “${./apps}” is a string, but it’s a special-case string (/nix/store/somelonglonghash-apps) which is handled in a way that ensures presence of the store object this string references on the target machine.

14

Me too, I was surprised that Nix doesn’t copy file to store. But a quick experiment in nix repl shows exactly this behavior:

nix-repl> ./readme.rst
/home/danbst/dev/imedicare/meta-servers/servers/readme.rst

nix-repl> builtins.toString ./readme.rst
"/home/danbst/dev/imedicare/meta-servers/servers/readme.rst"

nix-repl> "${./readme.rst}"
"/nix/store/md6z06jw7hcjbn6rihcr95xc4nphmw9l-readme.rst"