Skip to content

Runtime import#561

Merged
gviejo merged 5 commits intodevfrom
runtime-import
Feb 24, 2026
Merged

Runtime import#561
gviejo merged 5 commits intodevfrom
runtime-import

Conversation

@BalzaniEdoardo
Copy link
Collaborator

@BalzaniEdoardo BalzaniEdoardo commented Feb 20, 2026

Summary

  • Move heavy imports (scipy.signal, scipy.spatial, xarray) inside functions to reduce import pynapple time
  • Import time reduced from ~920ms to ~390ms (~2.4x faster)

Background

As developers, we discussed using lazy_loader.load from scientific-python/lazy_loader but decided against it for this package because:

  1. Early adoption risk - The tool is relatively new and may not be mature enough for production use
  2. Limited gain vs. breaking change risk - The potential benefits don't outweigh the risk of introducing breaking changes to the import system

We recognize that the current pattern of moving imports inside functions is suboptimal (slight overhead on first function call, repeated import statements). However, we defer adoption of lazy_loader until it proves itself stable across the ecosystem. We already have it as a dependency in nemos, which will serve as a testing ground.

Changes

  • scipy.signal imports moved inside functions in:
    • core/time_series.py (smooth, decimate methods)
    • core/_core_functions.py (_convolve)
    • process/filtering.py (already done)
    • process/spectrum.py (compute_mean_power_spectral_density)
  • scipy.spatial.distance.cdist import moved inside process/decoding.py
  • xarray imports moved inside wrapper functions in:
    • process/decoding.py
    • process/tuning_curves.py (already done)
  • Added test test_lazy_import_heavy_modules to ensure these modules aren't loaded at import time

Performance

Branch Import time (avg)
dev ~920 ms
runtime-import ~390 ms

Speedup: ~2.4x faster (57% reduction)

Additional fix

  • Fixed test_compute_1d_mutual_info and test_compute_2d_mutual_info tests in tests/test_tuning_curves.py
  • Changed np.testing.assert_approx_equal to np.testing.assert_allclose (lines 1591, 1663)
  • assert_approx_equal is for scalars only; numpy 2.4.2 (used in CI) enforces this strictly, causing TypeError: only 0-dimensional arrays can be converted to Python scalars

@codecov
Copy link

codecov bot commented Feb 21, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines Coverage Δ
pynapple/core/_core_functions.py 85.18% <100.00%> (ø)
pynapple/core/time_series.py 93.60% <100.00%> (+<0.01%) ⬆️
pynapple/process/decoding.py 92.43% <100.00%> (+0.12%) ⬆️
pynapple/process/filtering.py 91.83% <100.00%> (+0.17%) ⬆️
pynapple/process/spectrum.py 87.03% <100.00%> (ø)
pynapple/process/tuning_curves.py 97.35% <100.00%> (+0.04%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@h-mayorquin
Copy link

For a similar reason, in SpikeInterface, we keep an action on github measuring import time for the different modules:

https://github.com/SpikeInterface/spikeinterface/blob/main/.github/workflows/test_imports.yml

https://github.com/SpikeInterface/spikeinterface/blob/main/.github/scripts/import_test.py

This avoids regression.

With time, I have come to love the simplicity of delayed imports with just an import x on the spot where it is needed. Honestly, the downsides are 99 % aesthetical but you learn to get over them.

@gviejo gviejo merged commit 3f613b0 into dev Feb 24, 2026
16 checks passed
@gviejo gviejo deleted the runtime-import branch February 24, 2026 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants