Problems with Java version 8 (Oracle and OpenJDK) on CentOS 6.9 - glibc issue?

Help
1

I’ve been trying to build some Java projects on CentOS 6.9 (Red Hat kernel version 2.6.32-696) using both Maven and Gradle, with both Oracle JDK 8 and OpenJDK 8, and all combinations fail with issues which I suspect are related to trying to use a newer glibc on such an old kernel.

The following program comprises a minimal test case which fails with both JDK versions:

import java.io.IOException;
import java.net.ServerSocket;

public class Main {
    public static void main(String[] args){
        try {
            ServerSocket socket = new ServerSocket(32751);
            socket.accept();
        } catch (IOException ex) {
            ex.printStackTrace();
        }
    }
}

With OracleJDK, I get the following error message:

library initialization failed - unable to allocate file descriptor table - out of memoryAborted (core dumped)

With OpenJDK, I get the following error:

java.net.SocketException: Socket closed
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
        at java.net.ServerSocket.implAccept(ServerSocket.java:545)
        at java.net.ServerSocket.accept(ServerSocket.java:513)
        at Main.main(Main.java:8)

Running strace against each of these shows a call to “prlimit64” which fails with ENOSYS. Using the system-wide JDK (OracleJDK 1.8.0_111), which runs the program successfully, I see a call to “getrlimit”, which succeeds.

How should I proceed from here?

I’m using nix 2.0 (nixUnstable from channel 17.093238.e984f9e48e1), with each Java version from channel 18.03.132275.2352d46904d. Please let me know if any other information is required.

2

According to Stack Overflow prlimit64 is used by 32 bit systems, which seems odd.

3

From http://man7.org/linux/man-pages/man2/getrlimit.2.html:

Also, the NEWS file from glibc has the following entry:

Perhaps the existing patch to glibc to allow it to run on this kernel is not sufficient?

4

It’s quite likely, because of the RedHat LTS policies, CentOS tends to inherit very old kernels patched to oblivion.

Is there any more recent kernel available in the CentOS package set that you could upgrade to?

5

Short answer: Yes, and we have machines running newer versions of CentOS, so this is the right long term solution to the problem.

Longer answer: I would like to explore introducing nix at work. A prerequisite to this is being able to use a nix-versioned JDK on our CI/CD machine, which, unfortunately, hosts far more than it should (CI/CD, artifact storage, and version control) and runs on Centos 6.9. I was hoping for a solution which would allow me to get nix working for our use case without having to bother anyone else, so I can convince people of its value by demonstration.

After posting this, I found that one can use the LD_PRELOAD environment variable to point to a shared library that can intercept system calls. Using that to point to a shared library compiled from the following worked for me:

#include <sys/resource.h>
#include <unistd.h>

#define __NR_getrlimit 97
#define __NR_setrlimit 160


int prlimit(pid_t pid, int resource, const struct rlimit *new_limit,
            struct rlimit *old_limit) {

  long getrc = 0;
  long setrc = 0;

  if(old_limit != NULL) {
    getrc = syscall(__NR_getrlimit, resource, old_limit);
  }

  if(new_limit != NULL){
    setrc = syscall(__NR_setrlimit, resource, new_limit);
  }

  if (getrc == -1 || setrc == -1){
    return -1;
  } else {
    return 0;
  }
}

int getrlimit(int resource, struct rlimit *rlim){
  return prlimit(0, resource, NULL, rlim);
}
int setrlimit(int resource, const struct rlimit *rlim){
  return prlimit(0, resource, rlim, NULL);
}

Using strace still shows a call to prlimit64 which fails with ENOSYS, so I’m not sure what that’s about.

1 Like
6

Nice skills!

So just to be sure: the workaround is solving the problem but there is are some syscalls left with some unknown side-effect?

7

I would try adding older glibc into LD_LIBRARY_PATH or LD_PRELOAD. It’s supposed to have very good binary compatibility.

BTW, we’re quite stretching glibc specifically for RHEL 6 already: https://github.com/NixOS/nixpkgs/pull/32954/commits/87acb2b9fdb990ccb9f94a061b9bc21e4d31e0e1 but apparently it’s not enough in your case.

8

@zimbatm

Yes. I’m getting the outcome I hoped for (both JDK versions work as expected), but I’m still not entirely sure why - if LD_PRELOAD and strace work the way I think, I would expect either that the JDKs would work, and strace would show the syscalls supported by the kernel (getrlimit/setrlimit) being called, or that the JDKs would fail in the ways I saw previously, and strace would show the unsupported syscall (prlimit64). I don’t understand how the program is working while still showing the nonexistent syscall being called (and failing with ENOSYS).

@vcunat

Perhaps being impure and specifying the system glibc, or packaging the system glibc in nix (either from source or as a binary) is the better option. Who knows what other operations fail to work on a kernel this old? And the particular version of glibc I’m using explicitly states that it requires a Linux kernel of 3.2 or newer, so I’m not terribly surprised it doesn’t “just work” on CentOS 6.9.

9

RHEL / CentOS are special, because Red Hat backports lots of features to the old kernels. The link I’ve posted contains lots of details about what I’ve checked…