Disable reticulate auto-configuration in Nix-based interop derivations

[Copilot]

Reticulate 1.36.0 now auto-configures PYTHONPATH, which breaks rixpress’s hermetic Nix builds and causes import errors in Python↔R transfers. rixpress should always opt out of this auto-configuration while still allowing explicit overrides.

• Default reticulate env behavior
  • Always inject RETICULATE_AUTOCONFIGURE=0 into py2r/r2py derivations, with user-supplied env_var taking precedence.
• Interop API surface
  • Add env_var to rxp_py2r()/rxp_r2py() and document the default behavior.
• Build phase + documentation alignment
  • Ensure env exports are applied in reticulate build phases and note the behavior in README/vignette.

rxp_py2r(
  name = r_obj,
  expr = py_obj,
  env_var = c(RETICULATE_AUTOCONFIGURE = "1") # override default
)

Original prompt

Implementation Guide: Always Set RETICULATE_AUTOCONFIGURE=0 in rixpress

Problem Statement

Starting with reticulate version 1.36.0, the package automatically sets PYTHONPATH before loading Python to ensure modules are looked up in standard locations. While this helps in typical environments, it breaks rixpress's hermetic Nix builds.

The Breaking Change (reticulate 1.36.0)

From the reticulate changelog:

reticulate now sets PYTHONPATH before loading Python, to ensure modules are looked up in the same locations where a regular Python interpreter would find them on load.

Why This Breaks rixpress

1. Nix's Custom Directory Structure: Nix stores each package in isolated paths like /nix/store/<hash>-python3.12-numpy-2.2.0/lib/python3.12/site-packages

2. Pre-configured PYTHONPATH: Nix carefully constructs PYTHONPATH with all dependencies:

   /nix/store/5fwvl53qv4ic8vqqfiyiqh0kqzs18pb8-python3.12-numpy-2.2.0/lib/python3.12/site-packages:/nix/store/...
   

3. Reticulate's Auto-configuration Conflicts: When reticulate tries to "help" by resetting/modifying PYTHONPATH, it can:

   • Point to NumPy's source directory instead of the installed package
   • Reorder paths incorrectly
   • Override Nix's careful dependency resolution

4. Hermetic Builds: Each rixpress derivation runs in an isolated sandbox, so interactive shell environment variables don't propagate

The Error Manifestation

ImportError: Unable to import required dependencies:
numpy: Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and relaunch
        your python interpreter from there.

Proposed Solution

Always set RETICULATE_AUTOCONFIGURE=0 in any rixpress derivation that uses reticulate.

This tells reticulate to leave the Python configuration alone and trust that the environment is already correctly configured (which it is, by Nix).

Implementation Requirements

Functions That Need RETICULATE_AUTOCONFIGURE=0

1. rxp_py2r() - Uses reticulate::py_load_object()
2. rxp_r2py() - Uses reticulate::py_save_object()
3. Any R derivation that uses reticulate - Via rxp_r() with env_var parameter
4. Quarto/RMarkdown with Python chunks - Via rxp_qmd()/rxp_rmd() with env_var parameter

Always Set Internally (Recommended)

Pros:

• Works automatically
• No user action required
• Prevents a common error
• Can still be overridden if needed via future env_var parameter

File: R/rxp_file.R

1. Update rxp_common_setup()

#' Generate the Nix Derivation Snippet for Python-R Object Transfer.
#'
#' @param out_name Character, name of the derivation.
#' @param expr_str Character, name of the object being transferred.
#' @param nix_env Character, path to the Nix environment file.
#' @param direction Character, either "py2r" (Python to R) or "r2py" (R to Python).
#' @param env_var Named list of environment variables (optional, for future use).
#' @return A list with elements: `name`, `snippet`, `type`, `additional_files`, `nix_env`.
#' @noRd
rxp_common_setup <- function(out_name, expr_str, nix_env, direction, env_var = NULL) {
  expr_str <- gsub("\"", "'", expr_str)
  base <- sanitize_nix_env(nix_env)

  # Always set RETICULATE_AUTOCONFIGURE=0 to prevent reticulate from
  # modifying PYTHONPATH in Nix's hermetic build environment
  default_env <- c(RETICULATE_AUTOCONFIGURE = "0")
  
  # Merge with user-provided env_var if present (user values take precedence)
  if (!is.null(env_var) && length(env_var) > 0) {
    env_var <- c(default_env[!names(default_env) %in% names(env_var)], env_var)
  } else {
    env_var <- default_env
  }
  
  env_exports <- build_env_exports(env_var)

  r_command <- build_transfer_command(out_name, expr_str, direction)
  build_phase <- build_reticulate_phase(r_command, env_exports)

  snippet <- make_derivation_snippet(
    out_name = out_name,
    src_snippet = "",
    base = base,
    build_phase = build_phase,
    derivation_type = "rxp_r"
  )

  structure(
    list(
      name = out_name,
      snippet = snippet,
      type = paste0("rxp_", direction),
      additional_files = "",
      nix_env = nix_env,
      env_var = env_var  # Store for potential inspection/debugging
    ),
    class = "rxp_derivation"
  )
}

2. Update build_reticulate_phase()

#' Build Reticulate Build Phase
#' @param r_command Character R command
#' @param env_exports Character environment variable exports
#' @return Character build phase
#' @noRd
build_reticulate_phase <- function(r_command, env_exports = "") {
  phase <- sprintf(
    "export RETICULATE_PYTHON=${defaultPkgs.python3}/bin/python\n       %sRscript -e \"\n         source('libraries.R')\n%s\"",
    env_exports,
    r_command
  )
  phase
}

3. Update function signatures (future enhancement)

#' Transfer Python Object into an R Session
#'
#' @family interop...

</details>



<!-- START COPILOT CODING AGENT TIPS -->
---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey).