so today i have encountered a very strange issue with my nix shell,
first off, i am trying to install python3.13 but no matter what i did, i end up with python 3.12.8
and then upon checking ‘pymilvus’, it has correct version as shown on the website here:
https://search.nixos.org/packages?channel=unstable&show=python312Packages.pymilvus&from=0&size=50&sort=relevance&type=packages&query=pymilvus
however, the version that was actually installed was the updated version ‘2.3.6’. I do not understand why because this package was provided with both python3.12 and python3.13. how could this be?
any advice and ideas? thank you.
with import <nixpkgs> { };
let
unstable = import (fetchTarball https://nixos.org/channels/nixos-unstable/nixexprs.tar.xz) { };
pythonPackages = unstable.python3Packages;
in pkgs.mkShell rec {
name = "impurePythonEnv";
venvDir = "./.venv";
buildInputs = [
# A Python interpreter including the 'venv' module is required to bootstrap
# the environment.
# pythonPackages.python
pkgs.python313
# This executes some shell code to initialize a venv in $venvDir before
# dropping into the shell
pythonPackages.venvShellHook
# Those are dependencies that we would like to use from nixpkgs, which will
# add them to PYTHONPATH and thus make them accessible from within the venv.
# pythonPackages.numpy
python313Packages.numpy
# pythonPackages.requests
# pythonPackages.pygobject3
# python311Packages.weasyprint
# python312Packages.stdenv
python313Packages.stdenv
# this is from unstable channel
# python312Packages.llama-cpp-python
# python312Packages.magic
# python313Packages.mypy
# what the fuck with this stupid fucking naming fuck
# python312Packages.python-magic
python313Packages.python-magic
python312Packages.pymilvus
python312Packages.setuptools
# In this particular example, in order to compile any binary extensions they may
# require, the Python modules listed in the hypothetical requirements.txt need
# the following packages to be installed locally:
# gobject-introspection
gitRepo
gnupg
autoconf
curl
procps
gnumake
util-linux
m4
gperf
unzip
cudatoolkit
linuxPackages.nvidia_x11
libGLU libGL
xorg.libXi xorg.libXmu freeglut
xorg.libXext xorg.libX11 xorg.libXv xorg.libXrandr zlib
ncurses5
stdenv.cc
binutils
];
# Run this command, only after creating the virtual environment
postVenvCreation = ''
unset SOURCE_DATE_EPOCH
pip install -r requirements.txt
'';
# Now we can execute any commands within the virtual environment.
# This is optional and can be left out to run pip manually.
postShellHook = ''
# allow pip to install wheels
unset SOURCE_DATE_EPOCH
export CUDA_PATH=${pkgs.cudatoolkit}
# fixes libstdc++ issues and libgl.so issues
# export LD_LIBRARY_PATH=${stdenv.cc.cc.lib}/lib/
# did not know about this could be possible:
export LD_LIBRARY_PATH=${stdenv.cc.cc.lib}/lib:/run/opengl-driver/lib
# enable CUDA? currently 'is_cuda' returns false
# export LD_LIBRARY_PATH="/run/opengl-driver/lib"
# export LD_LIBRARY_PATH=${pkgs.linuxPackages.nvidia_x11}/lib
export EXTRA_LDFLAGS="-L/lib -L${pkgs.linuxPackages.nvidia_x11}/lib"
export EXTRA_CCFLAGS="-I/usr/include"
'';
}
FYI,
even after i specifically tell the version of python to be downloaded, which is 3.13, when everything is done, i still have 3.12.8
this clears things a lot thank you. but this happened unfortunately:
error: mypy-1.11.2 not supported for interpreter python3.13
sorry for being such a noob, in my latest attempt i have this:
with import <nixpkgs> {};
let
stable = import <nixpkgs> (fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-24.11") {};
unstable = import <unstable> (fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-unstable") {};
in
pkgs.mkShellNoCC {
packages = with pkgs; [
(python313.withPackages (ps: with ps; [
unstable.pymilvus
unstable.mypy
]))
# pymilvus
# unstable.mypy
curl
jq
];
}
it returns some syntax errors:
unpacking 'https://github.com/NixOS/nixpkgs/tarball/nixos-unstable' into the Git cache...
error:
… while calling the 'derivationStrict' builtin
at <nix/derivation-internal.nix>:34:12:
33|
34| strict = derivationStrict drvAttrs;
| ^
35|
… while evaluating derivation 'nix-shell'
whose name attribute is located at /nix/store/wlicjyms3s5pnzi3hmsaysn000rmc6ls-source/pkgs/stdenv/generic/make-derivation.nix:336:7
… while evaluating attribute 'nativeBuildInputs' of derivation 'nix-shell'
at /nix/store/wlicjyms3s5pnzi3hmsaysn000rmc6ls-source/pkgs/stdenv/generic/make-derivation.nix:380:7:
379| depsBuildBuild = elemAt (elemAt dependencies 0) 0;
380| nativeBuildInputs = elemAt (elemAt dependencies 0) 1;
| ^
381| depsBuildTarget = elemAt (elemAt dependencies 0) 2;
(stack trace truncated; use '--show-trace' to show the full, detailed trace)
error: expected a set but found a string with context: "/nix/store/j33wzkzndh41cyyy7i18bqm1srlv84cq-source"
same error after removing the unstable here:
with import <nixpkgs> {};
let
stable = import <nixpkgs> (fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-24.11") {};
unstable = import <unstable> (fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-unstable") {};
in
pkgs.mkShellNoCC {
packages = with pkgs; [
(python313.withPackages (ps: with ps; [
unstable.pymilvus
mypy
]))
# pymilvus
# unstable.mypy
curl
jq
];
}
error: mypy-1.11.2 not supported for interpreter python3.13
so far i havent been able to upgrade python into 3.13 yet. this is very frustrating.
sry i copied code before i removed both, after the removal of both this is the error message yield latest.
error: mypy-1.11.2 not supported for interpreter python3.13
thank you very much, I have made slight changes to include python313 and at least now the python version is correct,
{
pkgs ? import (fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-unstable") { },
}:
pkgs.mkShellNoCC {
packages = with pkgs; [
# install specific version of python
python313
# install specific version of python packages
(python313.withPackages (
ps: with ps; [
requests
# pymilvus
python-dotenv
]
))
];
}
however, the package pymilvus is still the problem:
removing build/bdist.linux-x86_64/wheel
Successfully built pymilvus-2.5.3-py3-none-any.whl
Finished creating a wheel...
/build/source/dist /build/source
Unpacking to: unpacked/pymilvus-2.5.3...OK
Repacking wheel as ./pymilvus-2.5.3-py3-none-any.whl...OK
/build/source
Finished executing pypaBuildPhase
Running phase: pythonRuntimeDepsCheckHook
Executing pythonRuntimeDepsCheck
Checking runtime dependencies for pymilvus-2.5.3-py3-none-any.whl
- milvus-lite not installed
error: builder for '/nix/store/xncgn9yxiazg4ajmv4xxwzmfx248h8np-python3.13-pymilvus-2.5.3.drv' failed with exit code 1;
last 25 log lines:
> adding 'pymilvus/orm/iterator.py'
> adding 'pymilvus/orm/mutation.py'
> adding 'pymilvus/orm/partition.py'
> adding 'pymilvus/orm/prepare.py'
> adding 'pymilvus/orm/role.py'
> adding 'pymilvus/orm/schema.py'
> adding 'pymilvus/orm/types.py'
> adding 'pymilvus/orm/utility.py'
> adding 'pymilvus-2.5.3.dist-info/LICENSE'
> adding 'pymilvus-2.5.3.dist-info/METADATA'
> adding 'pymilvus-2.5.3.dist-info/WHEEL'
> adding 'pymilvus-2.5.3.dist-info/top_level.txt'
> adding 'pymilvus-2.5.3.dist-info/RECORD'
> removing build/bdist.linux-x86_64/wheel
> Successfully built pymilvus-2.5.3-py3-none-any.whl
> Finished creating a wheel...
> /build/source/dist /build/source
> Unpacking to: unpacked/pymilvus-2.5.3...OK
> Repacking wheel as ./pymilvus-2.5.3-py3-none-any.whl...OK
> /build/source
> Finished executing pypaBuildPhase
> Running phase: pythonRuntimeDepsCheckHook
> Executing pythonRuntimeDepsCheck
> Checking runtime dependencies for pymilvus-2.5.3-py3-none-any.whl
> - milvus-lite not installed
For full logs, run 'nix log /nix/store/xncgn9yxiazg4ajmv4xxwzmfx248h8np-python3.13-pymilvus-2.5.3.drv'.
error: 1 dependencies of derivation '/nix/store/p9r6j38ljqmlqp44aszfsi10990mlicp-python3-3.13.1-env.drv' failed to build
i wonder could this be the specific package issue or it is because i havent included certain dependency.
thank you so much.
as you can see, the results are with correct versions now.
alice7@nixos ~/.d/test-shell> pip show pymilvus
Name: pymilvus
Version: 2.5.4
Summary: Python Sdk for Milvus
Home-page:
Author:
Author-email: Milvus Team <milvus-team@zilliz.com>
License:
Location: /nix/store/26cbvhyg3h2l8ffvpds7g15rq3xdfzrc-python3-3.13.1-env/lib/python3.13/site-packages
Requires: grpcio, pandas, protobuf, python-dotenv, setuptools, ujson
Required-by:
alice7@nixos ~/.d/test-shell> python -V
Python 3.13.1
when doing python on nixos, would you recommend to install all python packages/dependencies within nix shell, to avoid using pip.
in particular if i were to merge this sample shell.nix with the new code, what should I do? because i might need .venv environment afterall:
with import <nixpkgs> { };
let
pythonPackages = python3Packages;
in pkgs.mkShell rec {
name = "impurePythonEnv";
venvDir = "./.venv";
buildInputs = [
# A Python interpreter including the 'venv' module is required to bootstrap
# the environment.
pythonPackages.python
# This executes some shell code to initialize a venv in $venvDir before
# dropping into the shell
pythonPackages.venvShellHook
# Those are dependencies that we would like to use from nixpkgs, which will
# add them to PYTHONPATH and thus make them accessible from within the venv.
pythonPackages.numpy
pythonPackages.requests
# In this particular example, in order to compile any binary extensions they may
# require, the Python modules listed in the hypothetical requirements.txt need
# the following packages to be installed locally:
taglib
openssl
git
libxml2
libxslt
libzip
zlib
];
# Run this command, only after creating the virtual environment
postVenvCreation = ''
unset SOURCE_DATE_EPOCH
pip install -r requirements.txt
'';
# Now we can execute any commands within the virtual environment.
# This is optional and can be left out to run pip manually.
postShellHook = ''
# allow pip to install wheels
unset SOURCE_DATE_EPOCH
'';
}