Skip to content

add documentation for pydefix in virtualenv #365

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
glesur wants to merge 1 commit into
base: develop
Choose a base branch
from
+14 −0
Open

add documentation for pydefix in virtualenv #365

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions doc/source/modules/pydefix.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,9 @@ The python script `pydefix_example.py` initializes the initial condition of the
Troubleshooting
---------------

Pybind11 not found by cmake
++++++++++++++++++++++++++++

It during configuration stage, you get::

CMake Error at CMakeLists.txt:122 (find_package):
Expand All @@ -124,3 +127,14 @@ You can then exit the interpreter and set the pybind11_DIR environement variable
export pybind11_DIR=env/lib/python3.10/site-packages/pybind11

you can then run cmake which should be able to find pybind11, and compile the code.

Using pydefix in a virtual environment
+++++++++++++++++++++++++++++++++++++++
If you use pydefix in a python virtual environment (venv), pydefix might not detect your environment and fail to find pybind11 or other
python packages in your venv. This typically results in a python error "xxxx module not found" when you run Idefix with pydefix.

The fix is to force the following environement variables (replace XX by your python version) before launching Idefix:

```bash
export PYTHONPATH=$VIRTUAL_ENV/lib/python3.XX/site-packages:$PYTHONPATH
Copy link
Collaborator

@neutrinoceros neutrinoceros Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has much more consequences and can easily create a worse situation for a user, especially if they see the export as a suggestion to hardcode this line into a .bashrc or similar config file.
I'd strongly advise against this.

Copy link
Contributor Author

@glesur glesur Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see where it says this should be in .bashrc. My question would be: do you have a better proposition?

Copy link
Collaborator

@neutrinoceros neutrinoceros Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to be more constructive, I would suggest making pydefix a proper Python package if it makes sense (and it doesn't, there is a bigger problem to address here)

Copy link
Collaborator

@neutrinoceros neutrinoceros Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see where it says this should be in .bashrc.

I'm just saying this is typically where export statements from documentation / modifications to PYTHONPATH end up, and it's never1 the right solution.

Footnotes

  1. from the limited scope of my own experience ↩

Copy link
Contributor Author

@glesur glesur Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the problem is not due to pydefix (a c++ class in idefix), but to pybind11, which is already a python package. On Idefix startup, pybind11 does not necessarily pick up the python interpreter from the current venv.

Copy link
Collaborator

@neutrinoceros neutrinoceros Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I need to study how this is implemented for a couple minutes (possibly longer) before I can meaningfully contribute to this discussion, but I strongly feel like I should. Can I get back to you later this week ?

Copy link
Contributor Author

@glesur glesur Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for future reference:
the proposed fix comes from:
https://stackoverflow.com/questions/53480120/embedding-pybind11-with-virtual-environment

Similar issues may be found on the web, such as:
https://gitlab.kitware.com/autopybind11/autopybind11/-/issues/97

```