-
Notifications
You must be signed in to change notification settings - Fork 0
Description
Description
The plugin backend (persistent pytest worker pool) applies mutations by re-executing mutated source code into the target module's namespace via exec(). However, this approach fails when test files use from module import func style imports.
Root cause: When Python executes from mymath import add, it copies a direct reference to the function object into the test module's namespace. When the plugin replaces mymath.add with a new function object (via exec() into the module dict), the test's local add still points to the original function object. The mutation is invisible to the test. This is the same reason unittest.mock.patch requires patching where a name is used, not where it's defined.
Steps to reproduce
- Create a module
mymath.pywith a functionadd(a, b): return a + b - Create a test file that uses
from mymath import addand testsadd(1, 2) == 3 - Run
fest(defaults to plugin backend) - Observe ~10% mutation score
- Run
fest --backend subprocess - Observe ~90% mutation score
Expected behavior
Plugin backend should detect mutants at a similar rate to the subprocess backend.
Actual behavior
Plugin backend reports ~10% mutation score vs ~90% for subprocess backend. Mutations applied via exec() into the module namespace create new function objects, but test modules that used from X import Y still hold references to the old (unmutated) function objects.
fest version
0.1.2
Operating system
All platforms
Python version
All versions
Additional context
- Relevant code:
src/plugin/_fest_plugin.pylines 160-163 (_handle_mutant) - Workaround: Use
--backend subprocess - Possible fixes: walk
sys.modulesto replace stale references, re-execute test imports after mutation, or document as known limitation - Reported by https://www.reddit.com/user/DamnDoofus/