Hierarchy of module environments - why skipping global env and the package search path?

[amitramon]

When a regular package is loaded, the parent env of its imports environment is set to the base package namespace. The parent of this environment is the global environment, and that way code in a package can refer to objects in the global environment or in any attached package.

From what I understand about box (both from the article The hierarchy of module environments
and empirically, by explicitly checking), the 'imports' environment that is generated for a box module has the base package as its parent env. Since the parent env of the later is the empty env, code in a box module can only refer to objects of the base package, but not to object in any other environment.

While I guess this is by design and can see the rational behind this, the considerations behind this are not mentioned in the aforementioned article, and I'd appreciate to learn what they are.

On the one hand, preventing modules from accessing objects outside of the module (except for the base package) help making module code more robust (maybe we would even like to put this limitation on regular packages?), but on the other hand it is sometimes a bit restricting.

Thanks,
Amit

[klmr]

Hi Amit,

Your suspicion is spot-on: this is intentional. ‘box’ operates this way because it’s the correct way, and the behaviour of R packages almost certainly a bug which unfortunately can’t be fixed without breaking existing code.¹

I think you also misunderstand the intent of the behaviour of R packages. Per your explanation:

that way code in a package can refer to objects in the global environment or in any attached package.

But I don’t think giving packages access to objects in the global environment was ever the intent. This would actually be a terrible idea, and massively undermines code modularisation and loose coupling. Instead, the relevant section in R Internals says that

The enclosure of the base namespace is the global environment, so the search from a package namespace goes … to the standard ‘search path’. [emphasis mine]

This is less bad, but still not actually great (and completely unnecessary: packages should explicitly declare their package dependencies, and R CMD check will warn if they don’t).

Either way, the intent of ‘box’ is to fix issues with R’s package system, and this is one of them: leaky modularisation.

Let me reply to your closing remark:

On the one hand, [this behaviour] help making module code more robust … but on the other hand it is sometimes a bit restricting.

I completely agree with this summary but it’s phrased in a misleading way: because the gain in robustness is caused by adding restrictions, and restrictions are generally desirable for this reason. The less restrictive an API is, the more possibility for accidental misuse.

Anyway, the restriction doesn’t actually create any barriers. ‘box’ modules can of course access the contents of the global environment via .GlobalEnv (but they should almost never do that! — instead, they should export functions that allow the user to pass global values as parameters). And modules should never access attached packages; instead, they should themselves load the packages they require via box::use(). In particular, if a module needs to access the core R packages (besides ‘base’), it can load that package, or it can load (and attach) all of them at once to emulate the “conventional” R behaviour, via

box::use(r/core[...])

This is mentioned in the FAQ but ideally it would be documented elsewhere, I just don’t know where.

---

¹ Honestly, at this point it could (and should) probably be fixed. At worst it will break a few buggy packages, and R has never been shy about doing this in updates.

[amitramon]

Hello Konrad, thank you very much for taking the time to explain that in details. Like I said, I could see the rational of that behavior, but thinking of what you wrote made me completely convinced that this is indeed the right way to go. Also thanks for the practical examples that demonstrate how to work with box.

I'm still curious about the original intent of the R designers - the way I understand what is written in the R Internals is that it is intended that names are looked for in the global environment and then in the search path, but maybe I'm missing something. Anyway, that's not that important.

[klmr]

I'm still curious about the original intent of the R designers

Me too. One piece of the puzzle is that the original package mechanism in R evolved over time. For example, packages originally didn’t have namespaces. Namespaces were added later, and initially via an external package (‘namespace’). Consequently, the original packages had no NAMESPACE file and no Imports (only Depends). So they needed to be able to access packages loaded on the search path. Once namespaces were introduced, this was no longer necessary and became discouraged (but it’s still possible for backwards compatibility).

I don’t think accessing the global environment itself from inside a package was ever intentional: I can’t see a reason this would ever be a good idea. However, due to the way the search path is implemented in R, packages essentially have to have the global environment in their parent environment chain in order to access the search path (the search path could have been be implemented differently to avoid this, but it wasn’t).