Fix to allow pip install -e . to work seamlessly with newer pip/setuptools versions

[Crivella]

Fixes

• eb command fails outside of framework source folder #4451

The idea here is instead of relying on sys.path which might not include everything python knows how to import, we rely on python's import functions to determine if a module is importable or not and than pick the path from the imported module

[boegel]

This changes the semantics of import_available_modules, since no hard error was being raised before when the specified namespace didn't exist?

[Crivella]

For what i can see import_available_modules is always used to import all sub-modules of a specific namespace similar to doing a from NAMESPACE import *.

In particular for #4451 the failure would be obfuscated as something EB requires missing

I did not see a use in framework where this is expected to fail the import with it not being a problem.

If we really wanted to preserve the original behavior we could add a strict parameter to the call that default to False and set it to true everywhere this is used

[Crivella]

On a sidenote we could also modernize the fetching of the submodules using pkg_util.iter_modules following an approach similar to

https://www.tutorialspoint.com/how-do-i-import-all-the-submodules-of-a-python-namespace-package

EG

Importing installed package

(easybuild-dev) crivella@crivella-desktop:~$ python
Python 3.11.13 (main, Jun  4 2025, 08:57:29) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkgutil
>>> import importlib
>>> mod1 = importlib.import_module('easybuild.framework')
>>> mod1.__path__
['/home/crivella/Documents/GIT/easybuild-framework/easybuild/framework']
>>> l1 = list(map(importlib.import_module, sorted(list(_.name for _ in filter(lambda x: x.ispkg is False, pkgutil.iter_modules(mod1.__path__, mod1.__name__ + '.'))))))
>>> from easybuild.tools.utilities import import_available_modules
>>> l2 = import_available_modules('easybuild.framework')
>>> l1 == l2
True

Importing from current dir

(easybuild-dev) crivella@crivella-desktop:~/Documents/GIT/easybuild-framework/easybuild [feature-better_imports ≡ (5)]
$ python
Python 3.11.13 (main, Jun  4 2025, 08:57:29) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkgutil
>>> import importlib
>>> mod1 = importlib.import_module('framework')
>>> l1 = list(map(importlib.import_module, sorted(list(_.name for _ in filter(lambda x: x.ispkg is False, pkgutil.iter_modules(mod1.__path__, mod1.__name__ + '.'))))))
>>> from easybuild.tools.utilities import import_available_modules
>>> l2 = import_available_modules('framework')
>>> l1 == l2
True

With python 3.6 in a docker container

Ran pip install easybuild-framework with docker exec after launching the container

(easybuild-dev) crivella@crivella-desktop:~/Documents/GIT/easybuild-framework/easybuild [feature-better_imports ≡ (5)]
$ docker run --rm -it python:3.6
Python 3.6.15 (default, Dec 21 2021, 12:03:22) 
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import importlib
>>> import pkgutil
>>> mod1 = importlib.import_module('easybuild.framework')
>>> l1 = list(map(importlib.import_module, sorted(list(_.name for _ in filter(lambda x: x.ispkg is False, pkgutil.iter_modules(mod1.__path__, mod1.__name__ + '.'))))))
>>> from easybuild.tools.utilities import import_available_modules
>>> l2 = import_available_modules('easybuild.framework')
>>> l1 == l2
True

@boegel What do you think? Happy to add it in this or another PR (rewriting the 1-line version to be clearer)