Dealing with tests that require networking support

conej · June 26, 2021, 2:44am

I’m in the process of packaging a library in Nixpkgs, but the library in question’s test suite includes connecting to an external websocket server. While building the package on linux this test (and similar ones) fail because of the lack of network support in the sandbox. As an example here is one of the tests that fail:

github.com

microsoft/cpprestsdk/blob/bfe34877914fd280512c97d41aaa99e88564fd8d/Release/tests/functional/websockets/client/authentication_tests.cpp#L96


      
              if (msg.find("set_fail_handler") != std::string::npos)
              {
                  if (msg.find("handshake timed out") != std::string::npos || msg.find("Timer Expired") != std::string::npos)
                  {
                      return true;
                  }
              }
              return false;
          }
          
          
TEST(ssl_test)
          {
              websocket_client client;
              std::string body_str("hello");
          
          
    try
              {
                  client.connect(U("wss://echo.websocket.org/")).wait();
                  auto receive_task = client.receive().then([body_str](websocket_incoming_message ret_msg) {
                      VERIFY_ARE_EQUAL(ret_msg.length(), body_str.length());
                      auto ret_str = ret_msg.extract_string().get();

Jumping into the build environment confirms that the “echo.websocket.org” name does not resolve / attempting a connection with the IP fails because the network is unreachable.

I considered patching the test suite to remove these tests with external network dependencies for the Nixpkgs build, but I wondered if there is a better or preferred solution for this type of issue?

abathur · June 26, 2021, 4:41am

I haven’t had to deal with networking tests yet, but AFAIK disabling is correct for package tests.

Mindavi · June 26, 2021, 4:06pm

Disabling tests that do networking is typically done. Reaching out to the network is considered impure, so we don’t want that in nixpkgs.

peterhoeg · June 27, 2021, 6:08am

It is however possible to do NixOS tests that require a network connection. Depending on your use case that may do the trick.