I'm doing some research to understand the existing or desired use cases
of Nix for compiled-language development in industry. If your company
uses, or has seriously considered using, Nix for your development
environments or builds of a project you develop in a compiled language,
and you or someone at that company is willing to have a quick call or
email exchange to help me get a handle on your successes, pain points,
and wish lists, please let me know! I can be reached easiest at
shea@shealevy.com.
I’ve been trying to use it at a small java shop working with spark and data analytics, most of my troubles have been around getting it to work with maven. Will reach out when I have some bandwidth!
A lot of people don’t take into consideration how much effort was given to make a particular environment work with a given toolchain.
That said, your mileage will vary greatly for different toolchains. If you use good c/c++ practices with normal autotools/cmake/qmake conventions, it should be pretty easy. And it should be relatively easy on most compile languages. But I’ve also seen many code bases where they are full of hacks; and it will be painful to try and introduce nix, which is very heavy on purity.
I use Nix to isolate development environments for some projects, and by far the biggest pain points are channels and the absence of version info as first-class package metadata. Right now, I can easily say in my project’s default.nix that I rely on, e.g.: blas, boost, gsl, and icu for a C++ project. I cannot specify the versions I want without having to muck with old channels, dealing with packages which include versions in their names in some weird way, and generally getting a huge headache.
I don’t necessarily expect every single binary to be available for download along with every single one of its dependencies. Some caching would be great, of course, but even triggering a recompile of an old package version is really hard right now.
as long the derivation’s build steps didn’t alter significantly, you should be able to do package.overrideAttrs and set the version and source to be the version you need.
at least for boost, nixpkgs keeps onto each version
Sorry for refreshing this old thread. I used Nix currently with two languages: Python and Go, since you are interested in compiled languages. Actually go support was relatively good, the biggest problem (and this actually affects all languages) is no clear way to access app and dependencies that are in private repos. In go that involved modifying helper program that converted go.mod into a nix dependencies. I made it use fetchGitPrivate, which BTW was removed not long ago. The argument was that now there’s fetchGit in builtins, but it doesn’t map 1:1. Meanwhile goPackage function was introduced, which while quite nice, it made it impossible to use private repos. I think this is #1 problem when using nix in the industry (the only time this is not a problem is when your app doesn’t use any internal dependencies or you have a local source repo that doesn’t require authentication in LAN.
Yes. But this extends to infrastructure besides packages. We have started using Nix in some projects through the last 1.5 years, but I have found that you can only reliably do so if you pin nixpkgs to a particular revision or channel. Just taking a project that you haven’t used for more than a few months will often fail without pinning.
To give an example that affected some our our projects: hashes for buildRustPackage have changed twice in the last year or so. There are good reasons for this (cargo-vendor changes, migration to fetchCargoTarball), but such changes still break things, even more so when some people are on unstable and some on the latest release channel.
Then there are also packages which work at a given point in time and are then broken for long stretches (e.g. Tensorflow and PyTorch). Again, there are good reasons, people put a lot of effort in fixing the issues and often it takes some time to get the PRs in shape and reviewed. But it breaks things without pinning nonetheless.
The two big downsides of pinning are: (1) lack of security updates; (2) unless you have some monorepo with all your projects, the versions of nixpkgs that are used start to diverge, resulting in bloated binary caches + each repo having their own set of quirks to work around nixpkgs issues at a given time.
This isn’t meant as a harsh criticism. Nix offers a lot of benefits, such as reproducible development environments, much simpler CI pipelines (since it’s easy to pull in dependencies), etc. and is a big net win. But these are some issues we have run into in practice.
As a follow-up to this old thread, I’d like to link to the announcement I just made about Nomia, a new resource manager that generalizes Nix in ways that can address many of the concerns faced by compiled language users.