> [!seealso] See Also: > * [[Knowledge Base/Development/Languages/Python/Packaging/-Overview-]] > * [[pyproject.toml build backends]] # Build Isolation Both `pip install -e` and `python -m build` will build in isolation. The steps are: 1. Set up a temporary virtualenv 2. Install the build backend and its deps into the virtualenv 3. Construct the wheel 1. This is a special wheel for editable mode, set up as just metadata linking to the source tree `pip install -e` will then install that wheel in the outer environment. # Limitations * There are _no_ post-install steps available * Build backends can't locate packages from your normal virtualenv * _Might_ be possible to do this via a subprocess (`$VIRTUALENV` seems preserved), but this might be fragile/not future-proof. * Build backends can't install any development packages (no automated `pip install -r dev-requirements.txt`) * You _can_ copy those into the main dependencies for the wheel, in editable mode, though. Should be safe? # Testing Build Backends The build isolation means that any build deps in editable mode or installed either in a local virtualenv or globally will _not_ be picked up, leading to dependency errors: ``` ERROR: No matching distribution found for hatch-npm error: subprocess-exited-with-error × pip subprocess to install backend dependencies did not run successfully. │ exit code: 1 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. full command: /Users/chipx86/buildroots/rb6/bin/python3.9 /Users/chipx86/buildroots/rb6/lib/python3.9/site-packages/pip/__pip-runner__.py install --ignore-installed --no-user --prefix /private/var/folders/34/6wkbp94x5g79sbk8kfrscpzw0000gn/T/pip-build-env-1ohpnk5r/normal --no-warn-script-location -v --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- hatch-npm ``` There are a couple strategies here: ## 1. Set up local symlinks In [[pyproject.toml]], set: ```toml [build-system] requires = [...] build-backend = '...' backend-path = ['.local-packages'] ``` > [!tip] Don't forget the backend's dependencies! > Using `build-backend` and `backend-path` means the build backend's dependencies won't get installed. You'll need to list them in `requires` while testing. > > If you miss these, you'll get an error like `BackendUnavailable: Cannot import '<backend>'` Create `.local-packages/` and then symlink the root of the backend's _module_ (not the root of the source tree!) into the directory as an absolute path. For example: ```shell $ mkdir .local-packages $ cd .local-packages $ ln -s $HOME/src/my-backend/mybackend ``` Again, **make sure this is the root of the module!** It should contain a `__init__.py`, not a `pyproject.toml`. Then you should be able to install in editable mode or build packages: ```shell $ pip install -e . $ python -m build . ``` ## 2. Use --find-links (editable only) If you just care about editable mode (`pip install -e`) and you don't want to touch `pyproject.toml`, you can try the following: 1. Build actual wheels for the dependencies 2. Double-check that the right package names are in `[build-system].requires` in `pyproject.toml` 3. Use `--find-links` (`-f`): ```shell # The wheels should live in /path/to/dist $ pip install -f /path/to/dist -e . ``` # Debugging Missing Build Backends Figuring out why an in-development build backend can't be found is an exercise in frustration. This is in part because stages of the build are being run in your local known environment and in the isolated environment. There are a couple of useful places to start debugging this: 1. In `site-packages/pyproject_hooks/_in_process/_in_process.py`: This is run *inside* the isolated build backend. Look for where `no_backend` is being set. You can add some `print()`statements here, perform imports, whatever you need to introspect the environment. My strategy is to import the root of the build backend module and `print()` it to see what it's trying to import. You may also want to introspect `sys.path`. If `print()` doesn't do anything useful, try `assert False, your_value`. That will show up clearly in the output. 1. In `site-packages/pyproject_hooks/_impl.py`: This is used *outside* the isolated build backend, and consumes state from `_in_process.py`. Look for where `no_backend` is handled. You can explicitly print the traceback details referenced there, and also inspect the local environment to see if paths match expectations.